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. IDENTIFIERS 

.;ABSTHACT- ^ ' * ^ ^ * ^ . ^ 

This paper presents an analysis of data assessing the 
effectiveness of the Engelm^nn- Becker Direct Instruction Hodel (E-B) 
aimed at developing basic academic skills^ in disadvantaged 
primary-^grade children (with the specific; goal of enabling -fchese 
children to achieve grade- level performance by the end of ^rd grade) . 
/"The S-B model emphasize:^ small-group insttuction, using sequenced, 
daily lessons in l:eading^ arithmetic andilariguage desigfledNby . 
En,qeiaann and. published under the trade name DISTAB. Data on almost 
,12,000. disadvantaged students in E-B programs in^ 2 SyOregpn. school - 
districtis is presented .in tables and ^icc^omp^nying text. Half the 
' children had, started the programjin kind'ejigarten, half in 1§t grade. 

Achieve mejit was measured by the Wide Range Achievement Test, the 
• Hetropolitan Achievement Test and the siosson Intelligence Test. 
Be'sults. Indicate model effectiveness in;bqilding basic skills for a 
Jc4jie^^riety of disadvantaged students. j On all measure?, children 
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w^ere at or near national norms by the end of '3rd grade. Follow-up 
testing of 5th- and 6th- graders deocnsjtfated continuing 
effectiveness twp and three^ years after /the program in s^ite of 
V6ss^s again^ the norms. Testing lossei in reading comprehension 
after 3rd grade"*^re seen as due to the s^udden shift to an • 
unrestricted vocabulary in reading comprehension tests, and ^4th-grade 
textbooks. Report recommendations emphasx^ze an early start to basic 
skill instruction, systematic expansion of basic vocabulary and the 
Extension of follow-through proarams ^Until minimal adult competencies 
are -met.* (BF) . ' 
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\ ^ ^ ' ' / ' * " The Direct Jnstructipri Model* ^ ^ * > ^ • . 
' • . .'^ Wesiey C. Bfecker ^and*^Sief;fried Engelmann ' , . 

. • , ' ^ ^ University of Oregon * - " , 

• ■ • •< ' - r • . -. ' 

The Direct Instruction Model emphasi'ze« -small-group face-to-face instruction 

■. • r ■ ' ' - • . " " " V 

by av.teg.cher \isinf; carefully sequenced, daily lessons, in Reading, Arithmetic^, and 

' .• . V ■" • ■ - • , - ■ ■ - ■ i' . - " ' 

Language".. These programmed, lessons were designed fcy EngelmaAn using mode^rn learn- 

' ' ■ . ' . 

ing. principles and advanced programming strategies (Becjter, Engelmann, and Thomas, 

■ - ■ "■■ <- • ^ • . ■ • . . 

1975b,) and. are published by SF(A under the trade name DISTAR.^To achieve efficient 

t^eachinf, the teacher needs only to concentrate on effective presentation "techniques 

using the program materials. ^ . • ■ • 

• • '. * y \ 

, ■ In vthis report, we summarize the degree to .which the Direct Instruct ion ''Follow 

'V ' ■ • ' '■ ' ■ ' \ . 

-Through Model has been effective y examining the data on nearly 12,000 disadvan- ^ 
taged students *in E-^B 'Sponsored programs.; For a fuller description of the modk 
and, .the findings r-:er to Technic^ Report 76-I and the references provided therein 

. ' , Assumptions and Principles 

' ^ . ' • • ' . \ ■ . * 

The assumptions undeijlying the Engelmann-Becker Dire-ct Instruction Model are: 

1. M^lAchildren can be ^l^aught. 
' 2. ITieVearning of basic skills is essential to intelligent behavior and sh6uli^ 
be the. main focus of a com]Qensato^^educati6n program. . 

•3., The disadvantaged must be taught -at a faster rate than ' typically occurs il? * 
they ar^ to 'catch up' with their middle-class peers. 

We believe the goal of teaching all disadvantaged children basic skills at a 

V * ^ ' 

faster'rate can be accomplished by: . . 



This summary is adapted from: Becke?, W. C.and Engelmann, S., Analysis of 
Achievement Data on Six Cohorts of.Lov Income Children from 20 School Districts in 
the University of Oregon Direct Instruct i on Follow Through Model s Technical Report 
No. 76-1, prepared for the Office of Education. Eugene, Oregon: University of ^ 
Oregon, Follow Through Project / 1976. Program is funded by the U. S. O'ffice^of 
Education, Washington, D.^. 20202. ' " ' . < 



^ ^ - A ... , , . ^ - ... 

1 • The use of las'^uctional proCT^ms d^5if;rred to tW^h the general case ; When ^ . 
the teaching method can present only ^ome examples 'in a, set. and the child learns, 
to ,do all members of the set general cas3 has been taught and. an efficiency in 



\ teaching achieved. In the E-B Model*; programs designed to dc Just that are 
provided'^to the teacher, ' ' . ^ - \ ' ' 



2. Incr*easing the manpover in the classrQo mv People are 'the primary 'tpolL of 



instiruction. > By adding teache^' aides more, instruction. can occur, v i ' 
~3. Carefiilly structuring^ the dai l y propyam so that time is used to meet priority * 



teaching needs. Everyone knows what to do an^ when to, do it. 



^4. 'Using direct / small-group instructidfi . This is an efficient way to "individ- 
ualize instruction f or^ the ^non -reader. • ^ - / 

5. Using positive approaches to get and maintain , student attention, ^einf6!rce 

rood-- respbnses*, and correct* mi s,takes * . * . * ;^ , ' 

^ J ) V * * ' ' N • * ' ^ . V . ' ' ' 

6. Using carefu^ training and supervision to ensure that appropriate sk^,lls and 

procedures hav^'U)eeii proyided to the cla^sroom^ staff, . -/ , 

7. MonitqyiTY^tf >etudent progress/ with >iweekl5r^'ciXterion-r^ tests and 
repor'Cs^'oT^ Lessonjtau^ht to detect problems ^while there is time to correct them. 

. " TheseyComponent procedures have been used in d'esigning a model program that 

will teach basic akills, if it is followed. . * . ^ 

' Program Objectives ' 
* ^ ' * ^^^^'^ ' 

Nine DISTAR programs are at the heart of the model- They s^rve to specify the 

teaching objective's for students in Reading, Arithmetic, and Language., In DISTAR 

R^a^ing I. and II, the foctis is fiirst on decoding skills and then comprehension* 

Tlie children are "feught to say the soimds for letters and to sound out regul&p words 

' V . • / • " ; . -A 

Then irregulsLr sound words and letter combinations are added. As storey reading 

, - • • • . 

skills are built, 'the concern moves more to comprehension of what is read. In , 

Reading III, the children are taught to read for new information and to use that 



new irvformat^ioft. Most of ,the. .stbries have a ^science base to them which provides 
rules ^haV ceii be used to solve problems in afstronojny, muscle function,, or 
measurement.* Reading. Ill sjhould* prepare * the student to use. upper level textbooks. 

DiSTAR Arithmetic I teaches ba^ic addition and subtraction operations through 
'a problem-solving etpproach. Numljer .facts arfe memorized to speed up the process *and 

to set- the stage for more elaborate problems. \n Arithrael^it: II, the students are 

J ^. ' ' . ^ ' " ' / . ' • ' c ' • 

introduced to multiplication and fractions. Addition 'and subtraction are extended 

to column oi^erations, and a variety of measurement concepts involving tiiie^ money, 

length", and weight .are taui^ht. Ttie stuf'.ents are also taught niore n^3raber facts, 

• ' ^ . . . " ' ** . ^ / • , 

how to d^ive unknown f acts ,from known facts^, and^ how to work 3to-^ problems. In ^ 

Arfthmetid'III,. the ^students are ta^rght algebra, factoring, *and divi'sion, and^^the 



'traditional operatio^^ are .extended. 



€^ v^ISTAR Language ' I; an^ II 'teach Object Wes, object classes,' object properties 
and relatlonal/teT^k Children are taught tb make. whole statements and to appro,- 

priatel^p-^d logically describe the worl'd around them.' Language I and, II are 

' ^ ' . . . ' . . . ' 

basically, comprehension pror;j:ams, coupled with practice in language production. 

AmonWother^. things^, the students learn to d§al with conditionality, cau^jaliiy, 

multiple attributes, definitions, deductions, synonyms, and opposites. They are 

/" J ^ 
also taught how to ask questions in order to find out about , something. Language III 

«/ 

expands theological use of language (reasoning) .and teaches basic grammatical rules 

and structure. Many activities in Reading and Language are also geared to building 

. ■ 

writing and spelling skills. » ^ ^ • - 

J ' 
The major goal of th^e Direct Instruction Model is to give disadvantage^ 

children sufficient basic skills to compete with their more a4vantaged^)eers for 

higher e^'ducation and the opportunities available in our society. As an immedia^te ' 

go«l, we have used criterion of grade level performance (50th percentile) on 

major school achievement test batteries Jpy the end of third grade. We. are also 

J I 



Wncetrned that children. iGarrv a]^.s, efafts, social skills and valties, and ve 

•ericourapie instruction in the^^y^eas as suited to local conditions. We believe 

\ * • • * ^ 

fjjmiy that oi^r prpcedures.will build positive attitudes toward self, based on 

,.p*ersonal competence and positive treat^nent by others. However, a positiye self- 
' ' ' ' " # " ^ ' ^ ' i * 

concept IS "viewed as a by-product of p.ood* teaching rather than as a goal'that can 



'be. achieved* In th^^ abstract, ' • , . ,: - 

. * . • ^ ^ Installing; ttfe Program * W 



The Cont ract 

^ , • " \ 

■'As a sponsor, we contract with a school district to establish^nd monitor a 
program they have chosen. The contract includes budgetary requii ements" for staff,' 
materials, and facilities;, Job specificartions for staff; training requirements; 
monitoring. «&nd /^valjiation (procedures; and iJfethods for dealing with- cpntract vio~ 
-'lations. Initially, we trys-^eS school ^districts to adhere^y^ agreements , ^ We did * 
nof insist on writ^ ;.n contract's*. Problems .occurred -when'keyy^lbnnel were changed 
withput the new personnel being required to.keep thfe agreements made with tl^e 
sponsor. Now, we insist on acc'eptance erf contract^agreements before any. person ^ 
is hireU to work in" the» prograJh. ^ ' ^ v-^ 



Trai ling and' Supervision 

The goal of tr^ning is to provide the teachers (and parent aides X^th the 

' ' ■ * . . *' ■ • ^ 

skills required to t^ach small groups within the model* Teachers learn how to ^ 

group the children to get the most out of every child, how to use signals to 

coordinate group responding, how to present the DISTAR tasks, how to reinforce 

right responses, and how to correct mistakes. This training has usually been 

accomplished by a one- or two-week preservice workshop, continuing ^inservice ses- 

sions of about two hours a '^^eek, and classroom supervision, A nmber ot detailed 

procedural manuals have. been prepared for trainers and participants in training. 

With a structured teachj.ng system, it is possible to specify the teaching skills 



heeded. Training involves modeling and actual teaching practice with teachers 
taki^\turns playing the .students' roles. Teachers and aides learn by performing 
under • supervision ind gaining feedback on their performances. Just as their students 
dp, . A videotape. library which illustrates how to teach key tasksc:»in .the program has 
been helpful .for inservice training. Conep,e credit is given for inservice courses 

.in th6 basic reinforcement, and progyamminj^ principles underlying the program 

- / ^ ^ ^ \ ^ ' / 

(Becker, EngelmaJ\n, & Thoma5, 1975a, 1975b). Thi^ training has contributed to more 

n 

than 50' of our aidfea becoming certified teachers. ^ 

Classroom supervision is provided by consultants trained by the. sponsor. Many 
• V ' . ✓ • . . . • ' . ^ 

of these consultants are former teachers from^the local schools. Every trainer 

.must be able to demonstro-'te with children every aspect of the teaching x equirements 
' ^ » ■ • \ ' ^ . • 

• ^ \ . , . \ . • * . 

4 - . . * . A 

of the program. Thhy arte expected to spend '75^ of their worlcirig day in the classroom* 

' * \ t • \ * " " 

Manekgers ' * • V . i • \ * ' ' . 

Each project has a maii^ger assi*gned by Oregon.^ Managers mus£ b^ exner^nced *' 

• ^ V * ^ - ' • ^ J* 

teachers and supervisors who^now the classroom propejiures; anl assume resfTonsibility 

^ X . \ ^ // 

for all. phases of it* Managers- adapt schedules to local needs, monitor all phases 

\. ' 

of the program, identify priorilJie^/ and continually work for improvement* of imple-^ 
* " - \ * * « • • 

mentation. Managers spend luost of their on-site time, in the classrooms. 

Monitoring ' ^ 

The managers and supervisors prWide the first line of quality control ori^ 

\ 

teacher performance* The next line copies from biweekly reports of teaching activi- 
ties and of student progress on tests. \ To monitor student progress, paraprofess^onals 

\ 

are trained to give criterion-referenced tests in one area every two weeks. These 
brweekly reports ^how absences as. well as where. each group is in Reading, Arithmetic, 
and Language. The reports are used locally to regroup children, provide for_ tutoring,^ 
and/or to guide inservice training. They also let the sponsor ^know whether an 
ah^ceptable rate of progress is being made, and the quality cf that progress. Pro- 
jections of yearly progress enable, us to make program adjustments if the children 



fall behind. Thus, the program has huilt into it feedback loops at all levels to 
provide a "ba^si^s for corrective action at the time trouble occurs, 

^ ^The Childr en:- Served by the Prg^lect * 

: ip»^ 

The main' data described in this summary are based only on the Lov Income students 
from .?6 communities using the E-B Model. The communities include a cross-section of 
"poor". . America— -i^ral and inner-city Blacks, rural Whites, Mexican-Zta^ripans in 
Texas, Spanish-Americans in New Mexico, Native Americans iri/South' Dakota arid North 
Carolina, and a variety of ethnically mixed conmunitie^s.' Approximately 8,000 Low 
Income students are in the proeram at one time. The datVigrire presented coyer 
students who entered the" proRram between 1968 and 1973. wkhave -^nta on 5,922 
kindergarten-starting children and 5,565 first-grade stai-tiflg children in our main- 
analyses. Students ar^ included in this qinalysis if they meet the OSO poverty . 

guideline (called Low Income in this report), if they started the program at its 

\ > ' . 

earliest grade ier -x, and if tests are available at more than one point in time. ^ . 

/.»•"■ 

The data include students who may hpe left the progrtjn before completing it. 
■ For Wide Range Reading, the sample sizes are: / 

Pre-K Post-K Post-lst Post'^2nd Post- 3rd ^ - 

K -Starting : , , " 
2M5 5,181 1^,810 ^ 3,300- 1,988 

Pre-ls t Post-lst Post-2nd Post-3rd 

Ist-Starting : ^ ' ^ ' - ^ ^ " v*? 

~7 ^ 2 ,1*12 5 ,l60 U ,665 3,629 



This data' base provides a mstxim^im sample size for measurement of program 
impact. To check for biases in the outcomes because of changes in students^ over 
grades- (attrition), we have also analyzed year-to-Vearga.ins pn the same students, ' 
and fullptem pre-to-post gains (k-3 or 1-3) on the same students. These analyses 
do not materially change any conclusions except to make the actual gains^ about two- 
tenths. o]r a grade level , higher than those reported^ here- When the excluded non-Low 
Income children are added to the analysis (20% of our children), there is another 
slight increment in the level of performance. .Children who enter the program late, 
perfdrm a year lower on the average. This would be expected if the progx^am is 
important . 



vii 



The sample at the end of third grade is smaller because two groups that started 
in kindergarten and one group that started in first grade hav^ not yet finished. 
Th-^-^ry level testing is low because the first cohort was not pretested ^t all, 
and less than half of the next three -cohorts vere pretested. Examination of the 
sites with and without ;pretesting shows no detectable pattern that could Jjitroduce 

bias. Furthermore, -a comnarison of the partial data for cohorts 2/3, and Ij with 

^. , J'. 

the latter full data on cohorts 5 and 6 shows no systematic differences in entry 

.^scores. ' - . ' (} 

"^The Tests 



Thi-ee tests-were used. The Wide Range Achievement Tes^ (WRAT) provides 
measures of .Reading, Arithmetic, and Spelling, l^he Reading test is quite' reliable 
(.9X— .92) eSid has been demonstrated to reflect instruction which teaches decoding 
"skins. The Arithmetic test has questionable content validity for some levels of 
the test and lover reliability (.72— .80), bijt p^ovides^a gross measure of compu- 
tational skills. The spelling test "has reascinable reliability and (.'83) content 
■validity. 'The Metropolitan Achievement Test | (MAT)* provides measures of Reading, 
(comprehension and word knowledge). Math (computation, cohcepts, and problem 
solving). Spelling, Language (grammar), and at.>the "Intermediate Level, Science. 
The t^T has excellent reliability (.93— .96 for the tests reported) and adequate 
Validity as measures of some of our program goals. ' The Slosson Intelligence Test 
(SI,T)- is a short, individually administered test aimed at measuring what the Stanford 
Binet Intelligence Test measures (reliability .92, correlation with Stanford-Binet 
.93). We have- included it to measure some of the more general program goals tied 
to the logical use of language and reasoning skills. 

The vmAT and the SIT have been given to nearly ell students from the end of 
the first project year. The MAT was added at the end of first, second, and" third . 
grades beginning in Spring, 1972. Testing was carried out und'er the supervision of 



< 

the University of Oregon field staff by local teachers and aides. Careful tr^ia- 
inp, for lind wonltorinfr of testing was followed-* 

HI , 

* The Result s 

Norm-Referenced Gains 

A major goal of the pirect Instruction Itodel has been to F,ot Low-Income Follow 
Through students on a par wi^th national norms by the end of third grade. Table 1 
shows the degree to vninh this goal has been achieved for "Low Income students 
starting the program in kindergarten.* On the WRAT substantial gains against ^fche 
norms are present for all measures* These gains are displayed in Table 1 as 
percentiles on a standard-score scale.** The norm-referenced gai- :n WRAT Reading 
(decoding) is one-and-three- fourths standard devj.at ion units. The Low Income ^ 
students are close to a standard deviation above the norm on these important reading 
skills at the end of third grade. On WRAT Arithmetic $nd Spelling, large norm-, 
referenced gains ^ a demonsti ated and the students are functioning at the national 
norm median. 

We have no pretest for the Metropolitan Achievement Test meap'ores. However, 
on the basis of the WRAT data we wouldTestimate entry performance levels to be no 
higher than the 25th percentile and so we have used a Jagged baseline to illustrate, 
this probability in Table 1. The end pf third grade performances show the Low Income 
E-B ^tude^s to be at or near the national norm on all measures. Thejr fall a little . 

^Foi* the data in Table 1, statistics were computed with standard scores ffom 
tables providedby th^ test publishers. The mean standard scores were then converted 
to percentiles u^ng the publishers' tables. 

4 

^^Percentiles exaggerate differences near the mean and minimize those far from 
the mean. The graph has been drawn to overcome this problem. Assuming - normal 
curve, each horizontal line represents an increase of one-quarter standard deviation. 
However', the lines are labelled according to their percentile values. This provides 
the ease of interpretation of percentiles, as well as showing the magnitude of effects 
in standard score units • A one- fourth standard deviation gain is educationally 
important . 
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TABLE 



I Norm-RMerenced Gains on the Wide Range iPre-K to.Post3) 
and Post -3 Performance on/ihe Metropolitan for K- Starting, 
low Income. E-B Model Follow Through Students. 

WIDE RANGE METROPOUTAN * 

rotal Tot 

Reading I Arithmetic ] Speiiin^ 



84- 
77 
69 



50 
40 
31 

16- 



11 



7T^ 



RNdiSrMath spelling lapiuage 



PROJECT GOAL: ' 
National Norm Median^ 



PieK PDJt3|P»eR Post 3 Pie K Past 3 
Hsabef 9< Zmn9 2«5 J988 26JS »9S9 • 2520 1974 
Ptueoltles «8 83 19 54 [ 8 49 

StiQtfard Scores (SS! ' 

8 6 20 4 13 9 
3 99 ! - 09 3 JS 



StJfldard 0e«fati8Q «i SS 16 2 20 3 | 15 8 
Meis Grade £qr«3ie«ts 18 ^ 35 | 19 



|Nc Pretests) 



■4- 



P«st3 
1863 
40 



PfiSt 3 
1804 
52 



P«t 3!P8St 3 
1824j 1854 
4B f 50 



58 0 I n I j 62 8 I 68 8 

10 3 1 11 8 ! 113 j 13 2 
X3{ j 3 98 I 3 68 I 4 28 



3^ 



Short of the norm {UOth percentile) only on MAT Total Read£tig---a measure of; reading 
comprehension . 
Improvements over Cohorts 

In K-starting sites the data for cohorts 2 to C show progressive improvements 
on WHAT Reading and Arithmetic (ani to a lesser derree in SpellinpJ . The magnitude 



ERIC 



of the improvement is on the order of one-half standard deviation • The maj^ 
improvements occur at the kindergarten level and are maintained in later grades. 
It took some kindergarten teachers time to take seriously the task of teaching 
.academic skills to five^-'year-olds. 

The data from Ist-stgxting sites show a narrowing of the gap with national 
•norms, but with one less year to teach, the effects across, tests are about one- 
quarter standard deviation below those shown for the K-sites in Table 1. There is 
a signif ij-snt advantage in a compensatory catch-up program of having one more year 
to -teach when that time is used efficiently, 
" Controlled Withln-Project Comparisons 

V In 1970, /We started the Cherokee, North Carolina program simultaneously in 
kindergarten, fiz^st grade, and second grade. This provided a basis for comparing ^ 
the. performance of the children who had progressively more of the model. Since 
1970 we have also -ollected data on three additional cohorts, th.ese data permit a furthei 
assessment of the. progressive effects of better implementation (twp of these cohcJtts ^ 



^ have not yeh finisne^d third grade). 



Th$ Cherokee pro,lect xs located in the BIA school in the town of Cherokee. 
This is the ...only, school on the reservation.. With a constancy in student populatfon. 
and staffing, the study provides a rather' clear-cut indication of program impact. 
' The results from this analysis at the end of thirdr-^r^de are given in Table 2.^ ^ 

♦ 

For each measure. Table 2 shows a progressive improvement with increased years of 
tne .program or longer implementation of the program. l*he magnitude of the improve- 
ments from ^rly groups to later groups are in most cases more than one-half 
standard deviation. Only one of eight comparisons is not significant. 

In six other sites, we started. programs simultaneously in kindergarten and 
first gra^. On nearly every comparison^those starting in kindergarten performed 
significantly higher on the achievement tests at the end of third grade. The 
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Table 2 



\ Bnd'o^-J^ird Grade Percentiles for Cherokee Groups 

\, Having 2, 3^wd h Years of the E-B Follow Through Model 



Two Ye airs 



Three Years 



Four Years, Four YeEu:s, 



TOAT N 



99 



103 



. 85 



"Reading 
Arithmetic 



l45th 
3Uth 



79th 
l»5th 



79th 
55th 



8l4th 
50th 



MAT N 



109 



1Q2 



83 



Total Readiiig 
Total 'Math 

Language 

1 

Spelling 



36^h 
't7th 
ItOth' 
36th 



.UYth 
65th 
67th 
U2nd 



, 52nd 
68th- 
6'7th 

--Jj9th 



Tests comparing mean standard scores for two- and four-year 
groups, and three- and four-year- groups, were significant at the 
- .05 level or better in each case exCept the three- and four-year 
comparison for WHAT Reading. 



magnitude of the advantage averages, one-half standard deviation , or .6 to .8 

grade levels. The overall findings 'are consistent yith the previous comparison . 

made between K-starting and Ist-starting sites, but the vithin-site comparisons^^ 

show an>. effect tvice as large . Because of regional variations in populations and 

school procedures between K-sites and lst--sites, theie wi thin-site comparisons 

probably more truly reflect the potential gains of academic instruction during 

kindergarten Tor Low Income children. 

Fifth and Sixth Grade Follow-Up 

u * 

Approximately 700 g-B Follow Through fifth and sixth graders and 700 non- 
Follov Through comparison students were tested/ in seven sites at the end of fifth 
or sixth grade. The Wide Ran^e Achievement Test (Levels 1 and 2) and the Metro- 
politan^'Achi element Test (Intermediate Level), were given inmost cases. The 
compgirison groups were from schools having students with similsu:' backgroimds. 



However, wj-th samples of 35 to 100 students, variations in important background 
charac t^eristics are to be expected. For this reason covariance analysis was used 
to adjusti^-mean differ^ences on outoome measures for differences in student sex, , , 
father's education, mother's education, numbdr of siblings, income status, and 
ethnic group^ status. Four comparisons were made for most sites on the WHAT 
(Reading and Arithmetic using both levels of the test) and ten comparisons were 
made on the MAT (Word Knowledge, Reading, Total Reading, Language, Spelling, Math 
ComputaMbji, Math Concepts, Math Problem Solving, Total Math, and Science). The 
resul"fcs showed 53 significant differences out of a possible lk9 at the .05 3.evel 
using, a one-tailed test,. Of the 52 significant differences, 5Q fa-^o^-ed Follow^. 
Through ^and 3 favored the non-Follow Througn groups. Where significant effects are 
found, their magnitude averages about one-half standard devia^n. WHAT Reading 
measures showed significant differences in favor of Follow Through in ih of the 20 
comparisons.- MAT R' -.ding measures showed 11 out of 39 significant comparisons in 
favor of Follow Through. WRAT Arithmetic showed significant comparisons favoring^^ 
Follov Through in h of the 20 measures. MAT Math favored Follow Hirough sicrnificantly 
for .12 of ho comparisons. Science showed 3 significant differences ^ out of 10, 
Langu.age 2 out of 10, Spelling h out of 10. One* negative finding /was found for 
WRAT Level 1 Arithmetic. The other two negative findings were on MAT Spelling. 
Spelling was not a strong point of the program in the early years. 

Overall, the results are strongly supportive of an effect of the model which 
'is persisting (in the absence of special programs) two and three years later. The 
results in Reading (both MAT and WRAT) are especially encouraging. The reader 
should note that these results were achieved largely with children from Ist-starting 
sites. (Children from K-starting sites should do better. 

The level of student performance in the' years after Follow Th r ough aye in many 

r 

cases disappointing. There were sizeable losses against national norms from third / 
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1, or third to st)clh prrade. Losses wore especially p;reat .in bath (abbut 
three-fburths of,^ a standard deviation, 50th to 23rd percentile), and the reading 
comprehension (about one-fourth standard deviation, 35th to 25th percentile). 
Reading decoding skills were maintained, and apparently some schools taufrht 
spelling effectively in the intervening years. There is a clear implication that 



compensatory prc^ams cannot be Expected to maintain p;ains after. the proj^rams are 
stopped, unless all schools are prepared to provide effective teaching for every ^ 
- Qhlld in every p;rad e. This is cert,a,inly not the case today. 
Achievement of 'Students vith Lov^IQ's 

One of our concerns Is the development of procedures for teaching the hard-to-^ 
-teach Btudent.. A larp-e f^rouD of such children can be defined by their low IQ's. 
" W Haye.-examined the academic gains for our students with IQ scores below 80. 
Figure 1 shows comparisons for WRAT Reading gains for Lo w-IQ children and for All 
of our Follow Thi'^^r,h children. The data show, that Low-IQ chiiaren (me.an IQ = 73 
at pretest) gain more than a year on WRAT" Reading for ^ach year of instruction. 
On the average the gain i.i approximately 1.^2 grade levels each year, while the 
gain for. the All Group averages- 1.35 grade levels each^>ear;>^Gains in WRAT Arith- 
metic produce very similar results. The average gain for K-starting, Low-IQ 
students is -95 grade equivalents, while^the All Group averages l.QO. For Ist- ^ 
starting children, the average gain for the Lbw-IQ group is l.Oh grade equivalents 

and the All Group is 1.07. 

An examination of the number of lessons taught each year to the various groups 
shows that'iow-IQ children are xaught 30 fewer DISTAR lessons (^60 is avera,he) in 
kindergarten and 25 fewer in fir.st grade. After that the differential drops to 

. • / 

— . , 

*Ex.amination-of the gains in standard score units does not change tl^e 
implications of the 'data. ' ' 



Pre K - Post K 
•tow IQ'|1071 

All 12118) 



-1 



PcTst ■ Post 3 
^ ' Low la (40) . 

All (1374) 



-1 



Pre V Post 1 
low IQ 1305) 

- /^ ' aII (2113) 



Post 1- Post 3 
.. . Low IQ (408) 

■ ^ AH (3333) 



-1 



1.24 g^in 



K ■ Starting. 



^^X XXXXI 1.33 gain 



.K 



1XX^W9^ 1.33 gain 



1st • Starting 



2 



3 



'4 .5 



, 1st- Starting 

TO<y^XXxX >0^^2r52 gain * 



^^^^^^ 2.74 gain • 



1-' \2 . v3- 4 
WRAT Reading Grade. Norm 



.5 





K- Starting 

3.'32 gain ^ 




>^k>0<XXXXXX><XXXXXXXl 4.04 eain 


1 1 1 


i =^:^— 1 



Figure 1 Readin/. gains of low IQ disadvantaged children. (Shaded area 

indicates gain for the time period shoW to the* left of each chart.)' 

Low IQ « IQ o;^ 80 or less in Engelmann-Becker Follow Tlwrough^.Progrfyn. 
All ^-Ag^l^ children in Engelmann-Becker Follow Through Program. 
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less than 15 lessons (or % fewer lessons). The data i:nply that Low-IQ children . 

can be* taught a lofe more than is commonly as.^umed, 

. ' The children also flowed plains in IQ. The All Group 3hovs an avera/^e p,ain a? 

* i the end of third grade of about eight points, both K-starting and Ist-starting 
^ ^* *' 

^ , .groups- Discounting statistical regression effect,* the Low-IQ groups are esti- 

^* *^ 

mated tQ have gained between 8 and iH points on the averag^. 

^ . * , . " , 



\- 

\ 



*The true- score mean on pretest was estimated by using this formula x^, ~ \ * 
where x^. is the mean of the Low-IQ group in deviation units, and r^ is the reliability 



.coefficient (.92)- 




1 



1 • . — / 
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Conclusions 



!• The evidence shows that the model has been effective* in "building "ba^^R^ 
^kills 'and intelligence for a wide variety of disadvantaged students. On all 
measures our Lov Income'^ students are. aC|or nea^ national norms "by thp end^ of third 
grade. / . , " ' 

2. The evidence shows that the effects we hcR^e produced are educationally * 

... ' ' , f . i • .. ■. - 

significant^ in size and can s^till "be detected twq. and tliree years later ,i These 
significant folldv-up findings^ have "been shown largely .with children having the 
"benefit of only three years of the E-B program (ist-starting) * K-starting students^ 



may do better. The significant fifth and sixth ^rade effects are found in spite of 
losses against the, norms after leaving tMrd grade. 

3* The evidence strongly supports the value of beginning^ systematic instruc-r . 

; ■■■■■ . ■ . ' A 

tion in basic skills early. There is, a sizeable measurable benefit from Direct 
Instruction in kindergarten. Contrary $o .what has been attributed to oxir program 
by our critics, this has not been harmful tb children* The National Evaluation 

i 

measures of self-esteem (Coopersmith) and 6f parent and teacher attitudes (inter- 
views) for those in our model show that our teaching approach produces very positive 
attitudes as well as academic gains. , Our students are enthusiastic youngsters who 
enjoy learning and enjoy becoming competent. 

The evidence demonstrates that the Direct Instruction Model is clearly 
effective with Low-IQ children. 1?his is the group which has failed the most in 
the past. . ' - ^ 

f . The evidence shows systematic improvement for successive K-starting groups. 
. Better implementation of the program, as well as program improvements, have led to 

[ * 

better outcomes. Not yet seen m the data are the additional benefits likely to 
ensue from the second-generation of the DISTAR programs, which have been revised 
in the last ,two years . 



\ 
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Farther support for. these conclusions is provided by Rpsenshine (1976). In 
his r6v.i<gw of research on alassroom instruct ion, ^osensh^ne concludes that "in the" 
♦ cjrrretit research findinf^s on time, content covered, work groupings , teacher questions 
student responses, and adult feedback, |one*^^3 a general convergence of results on 
what might be labeled the direct instruction model (sometimes called a structured^ 
approach)." Rosenshine draws on the 'research of Stallings and M^skowitz .(197U) , 
Soar (1973), and 'Brophy and Evertson ■(197J») in drawing hib conclusions. Where the ' 
components b-f-the direct instruction model described earlier in this report- are 
present,, -positive gains in Reading and Math are -found. ■ . • J ^ 



Discussion ^ v 



. . 'two points in the data are expecially in heed of discussion. Why are we ab3e 

to reach national norms on all tests except reading comprehension, and why do the 
^ losses. against the norms occur from grades three to five or -three to six? The 

answers- to these questions are related. • • ■ 

' It is our conviction, based on a variety of evider>ce, that a good proportion 

of children from Low Income homes do not receive the .instruction in basic language 
^ concepts that the typical middle-class child receives. Sincebasic vocabulary is 

- a key element to success on intelligence tests, these children are likelj, to test 
lower on' such tests. - To a large extent, schools fail to systematically build basic 
vocab.ularies in students. Vocabulary in readers is controlled for the first three 

- grades', (and includf,..<5 only about 1500 words according to Chall) and then suddenly 
shifts \to a unrestricted adult vocabulary of about 15,000 words (Thorndike-Lorge ' 
estimate).. This St^dden shift is reflected in the performance of our students on the 
Metropolitan Elementary Level Test at the end of third grade. Reading comprehension 
is testel with a vocabulary that goes far beyond what has been typically taught in 



school. i 'This same vocabulary is also present in most fourth grade textbooks. It . 
is not surprising that children of poverty are likely to flounder at this time, or 



. - ■ \ , 

that our students should show losses betwe^n> the 'end of third ^grade and^ sixth grade 

when there is '"no Follow Through* The Low^ Income students do not get the home 

' * ^' . 

instruction in Language that miMle class children get. • ^ 

^ ^ / ' ' ' 

The losses in Arithmetic "between third "and sixth grade simply imply that schbol 

;pr^Oi^ams are not adequate (do not teq.ch) fpr^the instruction of Low Income'f^udents 
» • ' . *' 

at this level. The students in our ]^ograins showed they ^ could leai^n when they were 

'taught. ^ 

V If this analysis is correct, the implications are: 

1. - Begin basic skill instruction no later than , kindergarten for all educationally 
disadvantaged children. " ' ' 

2. .Revise school programs to systematically build Language 'comprehension through 

' , — ^ • "» 

a gradual" expanding of the admissible basic vocabulary. Do not leave this basic 

competency up to the home or chance. » / - - ^ 

3. Extend a Foll^ Through- type prograir- through the school years until at least' 
; minimal adult competencies are met. , — 
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CHAPTER I • _ // 



■ THE toEL.' ITS GOALS, AND THE TESTS 

• • Follow Through was initiated when, it appeared that gains made in 

. some .effective Head Start programs were being washed out in the early 

school grades. Originally planned as- an operational program, it was 
■ ■ ' . , ^ ; " ■ • ^• 

converted to a planned variation experiment when the initial funding 

. " level was cut from' $120 million tb- $15 raiillon. ' 'While ^uthbrizedvHander 
the Economic Opportunity Act,, the program was- administered :by the Office 

■ ■ 'of Education ^from the start. A pilot program was started -in 1967-68 in 
' 40 school districts. During this y6ar, the decision to find but what, 

works.ias made, and "program sponsors" were sought. Each Ebllpw Through 
community was asked to select from 'a set of predeveloped' approaches the 
one they would like, to work with. " - 

The initial'group of potential sponsors included Glen Nimnicht, 
David Weikart, Ira Gordon, Leonard Sealey, Marie Hughes, Don Bushell, 
Larry Gotkin, and Siegfried ^Engelmann. Consideration was given to 
^ the possibility of combining some sponsors into consortia. For example, 

the University of Oregon Diredt Instruction Model,. the University of ^ 
Pittsburg Individualized Early Learning Model, and the University of 
■'. Kansas Behavioral Analysis Model all relied on modern behaviorism for 
basic principles. However j, the differences between approaches were 
■ considerable. Pittsburgh and Oregon weje developing their own materials. 
One used individualized "instruction and th^ other small-group instruction 
as the main vehicle. Kansas relied almost entirely oh published materials. 
Thus^the idea of consortia was abandoned. 
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After preliminary discussions, 18 -sponsoring groups were invited to 
present* proposals to Follow Through communities for considcra^'lon. For- 
mally sponsored programs were begun in the summer of 1969. Eventually, 
165 school districts and 22 sponsors came to be Involved. 

By 1970, we at Oregon were working with 20 school districts, 
starting at the beginning level and adding a grade a year until the 
students .were In third grade. A new group of beginning level students \ 
has been Ridded to the study each year since 1968. In a given year, more 
than 9, COO students are in the Oregon sponsored program. 

The Follow Through .Guidelines call for a program of compreliensive 
^services including health care, social and psychological services, 
community involvement, and' an educational program. Sponsors^ could be 
responsible for any or all of these services. In most qases, however, • 
the communities assumed responsibility for health and nutrition programs; 
The focus in what fpllows is on the system built to install and operate 
the educational program. ^ ' 

BRIEF DESCRIPTION OF UNIVERSITY OF OREGON 

2 

Direct Instruction Jlodel ^ ' 
Engelmann defined the ba^ic problem faced in teaching disadvantaged 
children as one of devising a system to get more teaching going in the 
classroom. Only if disadvantaged children were taught more :Couia they 
learn more. 

Our Follow Through instructional system was dev^eloped having these 
components: 

1. Increased manpower in the .classroom. 

2. Structured daily routine* 

3. Daily programmed lessons. 

4. An efficient teaching method. 

5. Cont;lnuing training. 

6. Monitoring of progress of the children and the skills of the teachers 
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1. Increased nanpower in the classroom . _ When children cannot read, , 

■ ' ■ \ 

the primary means available for instructing them is by talking to them. 

If one is to get to every child and fully utilize the school day for , 

^instructionatafaster than ayei^age rate, .more than one teacher is required 

for 25 to SOy/hildren. Because of cost considerations, tvo- teacher 

aides were used." For the mo^t part the aides are parents of the poor 

children. It was our belief ithat parents who learned good ceaching skills 

would also be in r better position to facilitate their children's learning 

at' home. • 

^ 2. Structuring^ the daily program ^ Manpower by itself does not 
insure that more teaching goes on in the classroom. The organisation ■ 
of the -school day, a good progfam, and training are needed to effectively 

use the added manpower.. The classrooms are- set up so that the three 

- . • 
"teachers" are each working in booths (for sound control) with groups • 

of four to seven children. The teachers and aides become specialists ! 

in one of the three basic programs (Reading, Language, and Arithmetic) 

and a schedule is devised to fit 'each school's timetable to rotate the ; 

children through teaching groups and other activities when the children 

work on their own. Approximately thirty minutes is used for small group 

instruction in each subject area at Level I and II. At Level III, 15 

minutes of instructiorf is followed by thirty minutes of self-directed 



/ 
/ 

practice in workbooks. ■ / 



3. Pro prammed lessons . The instructional programs that are used 
in our Follow Through^lassrooms a;r¥ the DISTAlP programs (Reading, 
Arithmetic, and Language). These programs are potentially powerful- 
particularly with respect to teaching the general case. 
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4. An--e£ficient teachiaR method * The DISTAR programs are just , 
words on paper • In order ^ teach these skills, the teachers and ^des 
nust understand the concepts and operations they are teaching and nust 
have a number of basic teaching skills. These skills involve management 
of the children and .organization of the teaching miiterials SQ that both 
the children and the teacher are ready to work when they sic down fn^ 
an instructional group.- Beyond that, the teacher needs to know how to 
teach a task-rr-any task. 

The skills involved consist of knowing how to present the daemon- 
sdrations t^ the children; how to. use attention signals to get the 

i 

children to respond together (or individually) on cue; how to pace each 
task appropriately, quickly enough to hold attention, yet going slowly 
*when xequire^d to give the children "time to think"; how to use 
reinforcers effectively to strengthen correct responding; and how to 
correct mistakes in a way which permits all children to learn each 
task (criterion teaching) . ' ^ 

5. Training and supervision . The goal of training is to provide 

. the teacher with the skills outlined above. This is accomplished in a^.one 
or fcwo-week preservice workshop, continuing inservice sessions of about 
two hours a week, and through classroom supfervision. A number of 
detailed procedural manuals have been prepared for trainers and participants 
n training. The key is to know what the teachers should be able to do, 
ana" to devise procedures to teach the required skills. It should be 
recognized that precision in specifying and training essential teaching 
skills is only possible within a structured teaching system. 
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Classroom supervision is^rovided by consultants tralneti by the 
sponsor. .Many of these are former teachers from the local site. There 
is approximately one local supervisor for every two hunc^re^ children in 
the program. \ 

6. Monitoring . The management of the progress"" of -mojc^tthan 9,000 
children in 20 locations around the country requires a carefully designed 
monitoring system. 

Built into the DISTAR programs are teacher-given tests to check each 
new^aldrtL as it ie taught. To monitor child progress inderVHently of 
the teacher, continuous progress tests (criterion referenced) are given 
In. each area each six weeks by paraprofessionals at the Follow Through 
sites. Every two weeks test results in one area are summarized by child , 
on four-copy IBM forms, (vith names and numbers preprinted by groilp). 
These biweekly reports also sliow absences for the two-week period and 
show where each group is in each program. Copies of the reports go to 
the teacher, the supervisor, the Follow Throujgh Director, and our data 
analysis '•center. ' The reports can be used loyally to directly regroup 
the children. or to provide special remediation or acceleration. They 
also provide a basis for summary analyses gf progress for management 
by the sponsor. Trouble spots can be determined and worked on. 

Management reports are produced by computer to keep track of group* 
progress. Projections are made and compared with target goals for each 
group for the year. When, projections^ fall behind goals, adjustments in 
the program can be made at the site to attempt to reach goals before it 
Is too late to do anything about it. Management reports also keep^^^track 
of school calendars and absences so that it is possible to base . 

/ 

projections for each site on loca^ conditions that affect teaching days 
available. ' . / 
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PARTICIPATING' FOLLOW^ THROUGH PROJECTS 
This report covers data collected in the school years 1968-69 to 
Vjl^-lS in 20 Follow Through Projects sponsored by the University of 
Oregon, %irect Instruction Model. The sites, .«ntry levels and years 
covered by this report are shown in Table 1.1», 

Table 1.1 
Sites by Years a^d Entry Level* 



Sites 



L 



School Years 





68-69 


69-70 


70-71 


71--72 


72-73 


73r7A 


7A-75 


Brooklyn, NY, rbiJ/ 






K 


K 


K 


K 


K 


Cherokee, NC 






K-tI.. 


K 


K 


. X 


K 


Chicago, XL (Ogden) 




. K 


k 


K 


K 


(not T: 
eligil 


ttle I" 
3le) 


Dayton, OH 


1 


1 


1 


1 


1 


1 


1 


Dimmitt, TX 




1 


1 


\ 


1 


1 


Dropped out 
in 2/75 


E. Las Vegas, NM 


1 


1 


1 


1 


1 


1 


1 


E. St. Louis;: IL 


1 


1 


K-l 


K 


K 


K 


K 


Flint,. MI 




K 


K 


K 


K 


K 


K 


Flippln, AR 




K-1 


K 


K * 


K 


K 


K 


Grand Rapids, MI 


K 


K 


If 
K 


If 
K 


N6 Agreement 


Dropped 


Providence, RI 




K-1 


K 


K 


K 


' K ' 


K 


Racine, WI 


K-1 


K 


' K 


K 


K 


K 


K 


Rosebud, Tribe, SD 


1 


K 


K 


K 


K 


K 


K 


Smithville, TN 


1. 


1 


1 


1 


1 


1 


1 


Todd Cpunty, SD 


1 


K 


K 


K 


K 


K 


K 


Tupelo, MS 


, 1 


1 


1 


1 


X 


1 


1 


Uvalde, TX 


1 


1 


1 


1 


1 


1 


1 


Washington^ DC 


K 


K 


K 


K 


K 


K 


K 


W. Iron County, MI 


K 


K 


K 


K 


K 


K 


K 


Williamsburg County 




1 


1 


1 


1 


1 


1 



*K - K-Startlng, 1 - J st-Stfirting 
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OBJECTIVES OF TH£ EVALUATION 
, The evaluation was designed: 

1. To demonstrate that economically disadvantaged chi'ldren can be 
taught. so that they will show a rate of progress typical for aational noria 
groups ill reading and math for grades K to 3. (Priority: high)^ 

2. To demonstrate that economically disadvantaged children can be 
taught so that they show a mean IQ gain from entry grade through third 
grade, ^even though the usual expectancy is for a loss, on IQ tests. 
(Priority: middle) 

To demonstra'te that low IQ children (80 and below) from 
econo;aically disadvantaged homes can be taught at least a grade level a 
year . (Priority : high) 

4. Tp aetermine if any found gains can be shown to persist at 
fjrades 5 and 6. 

5* To examine the effects of introducing the model into Head Start* 

T'' how r>o:;r. ble r'^^laLion; of process ami oulconi<i rnear'ire:;. 

^ Designs 

A norm-- referenced design will he used as one approach to objectives 
1, 2, and 3. 

Control-groups designs are used for objectives 1, 2, 4, ^^n^ 
hxperinental approaches arc useu for objecllw r . 
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Measures 

1. Wide Range Achievement Test (WRAT) 

(a) Reading (decoding words) (R) ^ 
■ . (b) Arithmetic (A) 
(c) Spelling (S) 

2. Metropolitan Achievement Tpst (mI&) ' 
(a) Pritaary I, 

(1) Word Knowledge (WK) 

(2) Word Analysis (WA) ' 

(3) -Reading '(R) 

(4) Total Reading (TlO 

(5) ,- Total Math (TM) .-, 

^ (b) Primary II. 3 

(1) Word Knowledge (WK) 

- (2) Word Analysis' (WA) 

^ . (3) Reading (R) 

(4) Total Reading (TR) 

(5) spelling (S) 

(6) Language (L) 

(7) Math Computation (MCom) 

(8) . Math Concepts (MCon) 

(9) Math Problem Solving (MPS) 

(10) Total Math (TM) 

(c) Elementary -(Same scores as Primary 11 less Word Analysis.) 

(d) Intermediate (Same as Elementary plus Science.) 
3. Slosson Intelligence Test (SIT). 

' 4. Attendance. 

5. Lessons taught in programs (Reading, Arithmetic, and Language)- 

6. Criterion-referenced tests (Reading, Arithmetic, and Language) 
for the years 1970 'to 1973. 

7. Teacher-aide background form* 

8. Student background form. 

9. Individual site parent and teacher questionnaires. 

10. Teacher and aide evaluations by supervision. 

11. Student performance measures in experimental studies of instruction. 
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RELATION OF OBJECTIVES TO MEASURES 



Objective Measures 
Norm-Referenced Gains: 



In National In Site 
Evaluation 'Evaluations 



(a) Reading 



(b) Math 



(c) Supplemental 
Measures 



IQ Gains ; 

Instruction of Low 
IQ Students ; 

(a) Reading 

(b) Math 

Viftti'and Sixth 
' Grade Study ; 

(a) Reading 

(b) Math- 



(c) Supplemental 
Measures 



WRAT-Readlng '"^ (was ) 
MAT-WK, WA, Yes 
R, '!'R 



WRAT-Kath 



(was) 



MAT"-MConi, 

MCon, MPS, TM Yes 



WRAT-Spelling 

MAT-Spelling 

MAT-Language 

SIT 



WRAT-Reading 
MAT-Reading 

WRAT-Math 

MAT-Math 



WRAT-rReading 
I"1AT-Reading 

ITRAT-Math 
MAT-Math 



MAT-Scienca 

MAT-Language 

MAT-Spelling 



Effects of Head Start 
(Planned Variation) ; 

(a) Reading 



No 

Yes 
Yes 

No 



(was) 
Yes 

(was) 

Yes 



Yes 
Yes 



Yes 



Yes 



No 
Yes 
Yes 

Yes 



Yes 
Yes 

Yes 

Yes 



Appropriateness 
of Measure 

(5~very appropriate 
1-barely suited) 

5 (decoding) 
4-levels 1, 2 

3- level 3 (Comprehension) 

4- levels 1, 3 
2-level 2 ^ ^ 

4-levels X> 2,. 3 



5 

3 

4-levels 1, 3 
2'-level 2 
4 



5 (decoding) 

4 (comprehension) 

4 
4 



4 
3 
3 



WRAT-Reading 
MAT-Reading 



5 (decoding) 

4 (comprehension) 



Objective 
ih) Hath 

(d) Supplemental 
Measures 
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I^^ational In Site Appropriateness 
Measures Evaluation Evaluations of Measure 



WRAT-^Math 



MAT-Math ' ^ 
SIT-IQ gain^' 
MAT-Spelling 



4 
4 
3 



Relations of Process 
and Outcome: c 



Experimental ^ 
Studies 



Student Rate 

and Accuracy No No 

Time on Task No No 

Amount Mastered No No 



RELIABILITY OF STANDARDIZED MEASURES 
^ Wide Range Achievement Test (WRAT) 
From. the test manual: 







Split-harf 


Reliability 


Level I 


Age Group 


N 


Reading 


Arithmetic 


Spelling 


5 , 


200 


.981 


.966 


.971 


6 


200 


^.936 


.959 


.963 


7 


200 


.993 


.962 


.977 


8 


200 


.991 


.948 


.978 


9 


200 


.989 


.942 


.^77 


10 


200 


.990 


.948 


.981 



ERIC 
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iSese are probably comewhat Inflated because of. the ten-second time j 
limit per Item or because of the time limits on test sections. However, 
the time limits are very generous. and SRI alpha coefficients on a com-/, 
.bination 9f reading and math items ranged from^j,93 to ,97 for various 
samples.^ > 

The ABT report on Follow Through data for 1969-1972 gives internal 

consistency coefficients for partial WRAT tests for a random sample of 

" ' 4 
^ the' Follow Through test population (N « 6401) as follows: 

It- • 

Pre-^K. Testing Post-K Testing 

Reading .91 .92 

Arithmetic .63 ' .72 

Spelling .83 ^ .83 

How much these estimates are reduced by the shorter tests used is not 

knov^. However, the data are close to what we have found for similar^ 

^ groups on the full tests. 

Our analysis of a sample of 50 records for students pretested in 

the fall of 1970 and posttested in the spring of 1971, and the spring of 

1972 showed the following alpha coefficients: 

Reading / Arithmetic 

K Pretest Fall, 1970 .92 .79 

K Posttest Spring, 1971 .94 .80 

1st Posttest Spring, 1972 .94 .76 

The lower internal consistency reliabilities for WRAT Arithmetic 

». 

as used in our program are partly attributable to its lack of sensitivity 

o 

to what we teach. Tliis reduces the range of scores, particularly at level II. 
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Slosson Intelligence Test (SIT) 

The test manual states that test-retest reliability is estimated to 
be .92 for a population with standard deviation of 15, 
Metropolitan Achievement Test (MAT) 

The test manuals report thV following alpha reliabilities: 
Scales ' Test Level 

Primary 1 Primary II Elementary Intermediate 

Word Knowle'dge 
Word Analysis 
Reading 
Total Reading 
Spelling ^ 
Language * 
Math Computation 
Math Concepts 
Math Problem Solving 
Total Math 
Science 

5 

*From ABT Report, Vol* III 

VALIDITY INFORMATION ON STANDARDIZED MEASURES- 
Wide Range Achievement Test (WRAT) 

, 6 ^ 

Interpretability of Grade Norm Scores , Horst et al. nave 
indicated that grade equivalent scores for most norm-referenced tests 
are difficult to interpret because kids at different grade levels 
do not take the same test, and in relation to standard scores, the 
amount of learning representeil by a grade-norm change of one year can 
vary drastically for different grade levels. For example, in terms of 
standard scores, a grade-equivalent change from 2,0 to 4.0 on the MAT 
Total Reading Test is the same as a change in grade-equivalents from 
5.0 to 9.0. 



.88 


.93 


.94 (.94)* 


.92 


.90 


. .90 






.95 . 


■ .93 


.92 (.88)" 


.93 


.96 


.96- 


.96 


.9$ 




.94 


.96 (.97) 


•:96 






.93 (.89) 


.95 




.86 


.88 (.89) 


.84- 




.85 


.90 X.89) 


.88 




.88 


.91 (.88) 


.89 


.93 


.95 


.96 


.95 








.94 



1 
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The first criticism is not appropriate to WRAT Reading se res, 
since ail students from age 5 to 12 take the same test* However, there 
is ^^progressive change in the relation of standard scores to grade- 
equivalents. In progressing from grade 1 to^7, the standard deviation 
in grade-equivalent units incre^^s from 1.00 to 2*25 grades. Thitf 
Implies that the grade-equivalent score changes get rjelatively smaller 

.in standard score units with a progression in grade units. Because of 

» 

this problem, statistical analyses will usually be based on standard score . 
or raw score units, and grade equivalents will be used primarily as an 
aide to the , interpretation of the results. 

Sentitlvity to instruction . We have pi*li^hed several analysis 
showing systematic relations between progress on the WRAT Readinjt 

and Arithmetic tests and progress in the DISTAll Reading and Arithmetic 

"78 

programs for levels I to III. * The data imply that the WRAT .Reading 
Test is fairly sensitive to instruction in the DI3TAR programs (a 
valid .measure of one goal of the program — decoding skills) . The 
WRAT Arithmetic Test is also fairly sensitive to the program at levels 
I and III, but not to -level II. The lack of sensitivity at level II 
is due to the DISTAR program focusing on problems presented in rows ' 
(Important for later work in algebra), while the WRAT has a high 
percentage of problems presented in columns. At the end of level III, - 
this is no longer a problem, but the test again becomes insensitive at the 
fifth and sixth grade levels. Within year-level gains on WRAT Reading are 
significantly correlated with number of lessons taught in Distar Reading I and II 
Slosson Intelligence Test (SIT) 

The Slosson Intelligence Test is a short, easily scorable, test ^ 

» i 

designed to measure what the Stanford-Binet measures. For groups at 
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each age from 5 to 12, the correlations with the Stanford-Bine t Form 

* 

.LM were foiind to range from .94 to .98. However, the standard deviation 

for the groups tested were between 18 and 24, rather than 15. This would 

serve. to inflate the correlations. With appropriate corrections, the 

average yalidity coefficient is aboutV.93. 

Mccr'opdilitan Achievement Test (MAT)" 

• The basic statement about the validity of the MAT is contained in - 

the following quote from the Teacher's Handbook: 

The validity of an achievement test is defined primarily 
. ^in terms of content validity ^ A test has content validity if ^ 
the test items adequately cover the curricular areas that i:he 
test is supposed to evaluate. Since each school has its own 
curriculum, the content validity of Metropolitan Achievement 
' Tests must bfe evaluated by each school. It cannot be claimed - 
that the tests *are universally valid. To assist schools in' 
judging the content validity of the tests, the authors and 
publisher have prepared content outlines for the tests and 
"} describeu the procedures used in developing the test content. 

As this quote suggests, the validity pf any achievement test must 

be judged by examining* its content in relation to the program of 

instruction. We have examined the MAT at its various levtsls in relation 

to the DISTAR programs and finci the test to measure many things that 

the programs do not teach, and not measuring much that the programs do 

teach. The major problem that arises, is that reading skills (decoding 

and comprehension) are tested (esp«icially at the Elementary Level — end 

o*f third grade) using a vocabulary the students have not been taught. As 

one moyes from the Primary I and Primary II Levels of the MAT to the 

Elementary Level, reading becomes more and more defined as comprehending 

a vocabulary of unrestricted range (e.g., a full adult vocabulary). 

The children have to understand (as well as decode) stories about an 
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English .holiday (Guy Fawkes Day) and the "Gunpowder Plot" against the 
government, they have to read about Amazon Ants, Alexander Fleming *s 
discovery of penlcllln, architecture as an art form, museums with 
'' Egyptian artifacts, and a seated cat. Then they often have to answer 
questions based on conjecture about things not actually read about* 

Inhere are reasons to que^stlon the validity of MAT Eiementary Level 
Reading as a measure of the instruction occurring in our program. 

The following content descriptions of the MAT tests are taken from 

. ' ^ 9 ^ ' 

tK9 MAT Special Report 1971. / . — 

Word Knowledge , The Word Knowledge Test is Intended as, a measure 
of vocabulary. The "vocabiilary" of lnteres|i is that, which pupils 
encounter in their ordinary school work . . . knowledge of words, 
becomes less exclusively the province of heading anstructlon' as 
one moves up the grade scale. One begins to want to know how 
varied the pupil's vocabulary is — not just Whether or not he can 
read wc^Js that are in his speaking or listening vocabulary. Thus, 
the Word Knowledge items give a broad sampling of ^.words from various 
content areas (science, general experience, travel, etc.) and parts 
of speech. 

.Reading . The Reading Test ... .is intended a^ a measure of 
comprehension of connected discourse presented in a printed 
medium * . . there are four main types of questions which pupils 
must answer* (1) Pupils must be able to identify the main 
thougbt in the selection or select the best title for it. (2) 
Pupils must be alile to identify specific information in the selection 
or identify the literal meaning of the statements. (3) Pupils 
must be able to determine from the context, the meaning of an 
unfamiliar word or select the one of several known meanings of a 
word that is most appropriate in a given context; (A)^ Pupils . 
must be able to draw Inferences from the selection, to Identify 
unstated intentions and moods, and to see relationships between 
different points in the selection. These inferential-type items 
receive the greatest emphasis In the test. 

Spelling . Close attention was given to the appearance of words 
in spelling textbooks in selecting words for the Spelling Tests . . . 
words Wi^re included in the test to represent a variety of spelling 
errors * . . words are presented in dictated form. 
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Language . ... coverage is given to punctuation, capitalization, 
^ 'and usage of .words. A series of sentences is presented. Different 
parts of each sentence are underlined. Each underlined part may 
have one and only one type of error. The error may be in word 
usage, inadequate punctuation, improper capitalization, or there - 
may be no error at all ..... T Elementary Language" Test also 
Includes a test of sentence st^ ^.♦..The test of sentence sense is 
designed to assess the pupil's ability to recognize "telling 
sentences," *hsking sentences," and incomplete sentences. 

Mathematics Computation , •.. coverage is provided for simple 
number facts in the basic operations. The test includes horizontal 
and vertical notation, with more emphasis on the latter ... 

Mathematical Concepts . The Mathematics Concepts Test evaluates 
th6 pupil's knowledge of fundamental plrinclples and relationships 
4n mathematics. 

^ ^Mathematical Problem Solving . The' Problem Solving; Test attempts 

to evaluate the pupil's total developed ability in mathematics. 
. It demands reasoning with numbers and ojperations. It. presents , 
the pupils with everydayv problems in -consumer economics, practical 
measurement ... and other^ typical situations requiring numerical 
ability ... (1+) includes problems demanding use of all four 
fundamental operations and some multiple-step problems. .An attempt 
has been *nade to keep the voclg|ulary load and computational load 
• easy ... 

> 

Summary . : , 

The- E-B Direct Instruction Follow through Model is priniarily 
aimed at teaching basic skills better than typically has occurred in an 
attempt to catch children from Low Income families up with their ^ 
middle class peters by the end of third grade. Through the use of 
carefully programmed lessons, good planning for the use of time, 
extra "teachers", training, and monitoring of progress, the model 
seeks to reach its goal. While not explicitly stated as model goals, 
no .procedures are used which might hinder good social and emotional 
development. We expect our students to be happy as well as competent. 
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The major evaluation instruments used in the study are the Wide 
Range Achievement Test, the Metropolitan Achievement Test, and the* 
Slosson Intelligence Test (tb get at progress in language development) 
A summary of information on the reliability and validity of these ' 
instruments was provided in this chapter. 



' CHAPTER 2 

DATA COLLECTION AND ANALYSIS PROCED^JRES 

The stx:ategy which was adopted during the first year of Follow 
Through (1968-69) was to test each student at entry and each spring 
thereafter on the WRAT aad the SIT. Each of these tests takes about 20 
minutes and can be used with five-year olds who cannot read or write. 
Each is appropriate for a longer-^term, follow up of students as planned 
in Follow Through. They provided measures of reading (decoding), 
arithmetic, spelling, and general language compcitency (SIT IQ). In - 
the spring of 1972, the MAT was added to the testing schedule at the 
end of grades 1, 2, and 3. The MAT provided additional measures of 
reading comprehension, more extensive testing of arithmetic skills, and 
additional measures of language, spelling, and at the fifth and sixth 
grades, science. 

DATA COLLECTION PROCEDURES 

Who Was Tested f 

During the first year, -there was no pretesting. After that, 
pretesting occurred for most students. In the spring of 1969 only 
Washinp^con, D. C. (Nichols Avenue) was not tested because of a conflict 
with local testing. After that all sites were tested each year that they 
continued in the study. As noted in Chapter 1, Chicago (Ogden) was 
dropped from fui^ding in the fall of 1973, Grand Rapids, Michigan could 
not come to an agreement with the sponsor after the spring of 1972, and 
Dimmit, Texas dropped Follov: Through in February of 1975. 

> . , ■ 

3.-.) 
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•* * • " 

In* the spring of 1973, the SIT was given only to. students In 

entering and third grades because of a decision to cut down on testing 

' » " * . / 

costs. This decision was reversed the next yeqr. ^ Williamsburg was not 
;*iven the MAT In 1975 because of a decision to reduce pressures ar^^slng 
from testing In that site. Outside of these exceptions, evAy available 

\ A-.. * ■ • ' 

.Student enterltig the program throughr September 1973, was tested In each 
site, each year A 

Testing Procedures^ ' , 

Each year, the testing was guided by a manual of procedures and ■ 
a checklist set up before hand to provide step-by-step- infonnation. A 
copy of the 1975 manual is attached as Appendix A of this Technical Report, 
The manual of procedures was built around the test publisher^ manuals , 

g 

\ 



so as to be consistent with all standarolzation requirements. Training | 



began at Oregon (or Ulinois for 1969 and 1970) with the managers and 
supervisors who were to serve as the field supervisors for testing. Under 
the direction of Dr* Becker, a workshop for teat supervisors was held each 
year to go over in detail all procedures for setting up, giving, monitoring 
and checking of the testing program. 

On site, a similar trairlng program occurred for the local staff 
assisting in the testing. All teachers were trained to give the MAT 
^nd the aides were trained to monitor the students. The local teachers 
and aides were also trained to give one of these components of the 
t;esfc battery: 

1. Individual administration of the SIT. 

2. Oral reading and oral math parts of the WRAT (individual testing). 
3* Group administered parts of the WRAT. ^ 
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To prevent errors in subject identification, birthdates> etc., 
student names were printed on each test ai: Oregon and placed in manila 
•envelopes, by class^ ready for distribution. Extra tests were sent to 
accommodate new students and omissions. The examiner's names and date 
6t .testing were filled in on all !tests on site. Upoti completion of the 
tests, they were alphabetized within the class and placed back into the 
mat^la envelopes. A class roster was provided to record which students 
had taken the tests, were absent, or had dropped out the program. 

Testing was carried out monitored by the Oregon suplsrvlsors . 
Occasionally, monitoring reports indicated that the testing conditions 
vete not acceptable (e.g., the students w'dre given help by the tester) 
and the tests were discarded. During the first year all data from one 
site was discarded because of a failure to follow standardized procedures 
Over the years we have identified two classes where evidence of teaching 
test responses on the WRAT or SIT was detected. With longitudinal data, 
unwarranted help stands out clearly as non-uniform' jumps followed by no 
progress. 

In the* early years, tests were scored the day of the testing so 
that any problems might be noted and corrected>^on site. All tests were 
then restored by data clerks^at Oregon. Starting in 1973, MAT and SIT 
tests have be^erTco^puter scored at Oregon. ^ 

Stanford Research Institute administered MAT tests to a. small ^ 
sample of Follow Ihrough students after 1970. Approximately 500 third 
graders from fou.r sites were* involved (West Iron County, Flint, 
Brooklyn, Providence). To reduce duplicate testing, copiesSere made of 
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the Stanford test responses • A quality control check was used to 
Insure, accuracy. Personnel used in the data transferring were hired 
by the site director and trained locally. When the transferring was 
completed and passed checking, all tests were mailed to Oregon, 
Fifth and Sixth Grade Studies 

In addition to the regular spring testing, a study on fifth and 
sixth graders was conducted in 1975. Approximately 1,AOO students Wre 

involved in seven sites. Prior Follow Through students and control ^^groups 

\ 

of students with comparable background were tested. In mos^ cases, 
students in this follow-up study were given the MAT,, SIT, and WRAT 
test^. On the WRAT, both level 1 and 2 were given, although the 
spelling section was omitted. The spelling section on the MAT was 
considered Tficient* The Intermediate 'level MAT test was administered 
in most cases. A student. information form was filled out for all 
students. Prior Follow Through students were identified by a roster 
sent out by Oregon. Control groups were sought which were equivalent to 
the Follow Through students on factors such as income, parent education, 
ethnic group, and eligibility for the free lunch program. Under the 
supervision of project managers, local data staff and parent workers 
located students and filled out the student information forms. Personnel 
used for this testing were usually the same as those used for the regular 
spring testing. In larger sites, a few extra testers were bro.ught in 
from sites that had already finished their testing. In Tupelo the local 
district gave their fifth and sixth graders the California Achievement 
Tests, so the MAT was not given. The CAT reading scores were converted 
to Metropolitan equivalents by using the Anchor' Test Study conversion 
tables. 
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Other Information From Sites 

Student-Infonnatlon forms and Flnal-Report-of-Teaching forms were 
provided to the site data chiefs In the fall and spring, respectively. 
It was the data chief *s job to insure that such forms were returned 
completed. The Flnal-Report-of-Teachlng contained information on days 
completed in each Distar program (Reading, Arithmetic, and Language), 
a recoi:d by child of days absent, and a listing of late adds and early 
drops from the classroom. Bi-weekly reports of teaching and progress 
on continuous tests were sent to the sponsor by data clerks each two 
weeks . 

Again, to reduce errors, all reports were made on computer printed 
forms containing student names and identification numbers. 
Further Qualit y Control Checks 

Tests and other reports from the sites go through a routine screening 
for accuracy prior to being placed on IBM cards and fed to our computer 
file. A checklist of common errors is used for each kind of data. The 
first time the data go into the computer, logic checks are «ade where 
possible (e.g., " ^ ) and data falling outside of preset limits 'is 
Isolated and verified or corrected by our data clerks. Next, a preliminary 
report is prepared and sent to the sites with individual student data, 
classroom summaries, and grade level summaries. The sites and managers 
are asked to report to us any potential problems. After this the data 
are placed into the main data file. 

METHOD OF DATA ANALYSIS 
All children were assigned unique "numbers and their data stored on 
one continuous record. The available records were then coded by grade 
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and time o*f data collection to permit grouping together all children/ 

who had WRAT, SIT, and Final Reports of Teaching data from the sam4 

I 

time periods. ' » 

I . 

tor example, a child who was ijn the first grade in the^/all of 
1970, in the first grade in the spring pf 19/1, and the^cond grade 
in the spring of 1972 was assigned a code of 1F70 1S71/1S72* This 
method of coding permits analysis j>f the effects of retentions and 

skipping, since it isolates retentions clearly, e^.g., the code 1S71 

/ 

1S72 is obviously a retention^/ It also permits us to group children 
from the same grade levels^r cohort together if we wish,, even if 
their pattern of test rec^drds are different. Consider these codes: . 

Child 1 1F69 1S7(/ 2S71 

Child 2 1S70 2S71 3S72 

Child 3- r . . . 1S70 1S71 

Child A 1S70 3S72 

Each of these children is in ,the group (called a cohort), starting 
the Fall of 1969. By a proper combination of their records, a maximum 
number of children can be studied in an analysis of gain scores. Note 
that Child 3 is a retention. 
Treatment of Retentions 

The way in which children retained one or more grades are handled 
can have an effect on the interpretation of results. In control-group 
comparisons, retentions do not present a problem unless different 
retention policies are used for experimental and control groups. In a 
norm-referenced comparison, placement of retentions with their apemaces 
(keeping them in their starting cohorts) could produce an underestimate 
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of the program effectiveness. Placing them with their grademates could 
lead to an overestimate. As it turns out for the data to be presented, 
the two methods of analysis produce very similar results. We will 
therefore make the main oresentations with retentions grouped with grade- 
mates and later show how little the results change wher they are placed 
with agemates. 

K~S tar ting Versus Ist-Starting 

The coding procedure also involved a specification of K-starting 
and Ist-starting children, so that children starting in kindergarten, 
but first tested at the end of first grade, would not be confused with 
children starting in first grade. Since there have been numerous changes 
in K~8tarting and Ist-starting status because of the increased use of 
kindergartens, this specification was made individually by site for 
each data year. This identification allowed an analysis of the effects 
of the kindergarten-year on the outcome. 
'tains. Singles and Late-EnterXn^' Students 

Coded records were dividfe(^^ into *'gains" records, "singles" records^ 
and "late entering" records. \"Ga^i^ns" records were students who started 
the program at the entry level and who had test scores and Final Reports 
of Teaching for a^ le. st two testings * Iii the beginning cohorts, many 
"gains" students were not pretested. "Gains" students do not necessarily 
complete the program. "Singles" records come from students who have only 
one test in the file. "Late entering" records are from students who did 
not start the program at the beginning level. These were analyzed 
separately since they do not represent a full program impact. 




) 
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The "gains" records were analyzed in three ways: 

1. Suinmary statistics based on all available data. This gives 
the largest N. This data base is called MAX-N GAINS ^ 

2. Between-year gains based on the same students. This gives an^ 
inbetween N. This data base is called BETWEEN-YEAR GAINS . 

3. Pretest (kindergarten or first) to posttest (third grade) 

gaind>on the same students. This gives the smallest N. This 

data base is called PRE-POST GAINS , ' 

The lacer discussions of data for "singles" and "late entries" will ^ 

2 

rely on the data presented in our 1973 Technical Report. These earlier 
analyses include 3,600 of the present 4,883 records in the "singles" file, 
and 1,020 of the 1,926 in the late entering file. Because this is low- 
yi»ld informnr:ion, showing essentially the results previously reported, 
the data will be briefly described, but no,t presented. 

Standard computer programs were used to compute means, standard 
deviations, and other summary statistics used to describe the data. 

SUBJECTS 

All students entering the University of Oregon Engelmann-Becker 
classrooms from September 1968 through September 1973 on whom we have 
acceptable data records, are included in this study. There are 
21,485 records in the master file. Analysis of these records *is shown 
in Table 2.1. ' 



Table 2.1 
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Table 2.1 . 
Breakdown of Records in the Computer File 

RECORDS EXCLUDED HCM ANALYSIS , 929 

*^ 

Names but no data 536 

Bad Coders 6 * 

Head Start Planned Variation Only . 140 
Class entered above progrvjn 
(used as within site control 



groups) 247 

"SINGLES" ANALYSIS 4,883 

"LATE ENTERING" ANALYSIS •1,826 

All Kids Low-Income Kids 

K-starting 1,122 ^ 749 

Ist-starting 704 515 

"GAINS" ANALYSIS 13,847 

All Kids Low-Income Kids 

K-starting 6,995 5,922* 

Ist-starting 6,855 5,565* 

PRE-POST GAINS ANALYSIS 

All Kids Low-Income Kids 

K-starting 435 374* 

Ist-starting 1,208 1,080* 

TOTAL RECORDS - 21,485 



^Actual analyses will -always have smaller N's because of partially 
missing data. 
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ExcluHed Records 

Names but no data are most lit^^ly from children who entered the 
program without being pretesteXandjlef t before the first posttest. 
Bad code^ are records where an error in grade or date makes the code 
impossible, and where we have not been able to correct the error. The 
Head Start Only children participated in a sponsored Planned Variation 
Head Start but did not go on to Follow Through. Class entered above 
program students are students who were included in Follow Through but 
•did not go through the model program in kindergarten or first grade. 
Retentions and Skips 

In K-starting sites, there have been 350 retentions for one year 

(6.27. of Low-Income students in "gains" file), 9 retentions for two 

years, and 17 skips of one year. In Ist-starting sites, there have been 

606 retentions for one year (11% of Low-Incorae students in "gains" file), 

22 retentions for two years, and two skips of one year. 

Breakdown of Students by Site , 

.Tables 2.2 to 2.3 give the number of students in the "gains" 

analysis by site and cohort (retentions and skips with grademate) . 

Cohort 1 started in the fall of 1968; cohort 6 started in the fall of 1973 

A breakdown and ddscussibn of the "singles" and "late entering" groups 

3 

by site and cohort can be found in our 1973 Technical Report. 



Table 2.2 - 2.3 



Tables 2.2 and 2.3 permit the reader to see the constitution of the 
data base in terms of cohorts (starting years) and sites. The comparison 



Table 2.2 

H ioT All and Lov-Income K-Starting Students in the MAX-N-GAINS Analysis 
by Site and Cohort. Retentions with Gradeaates 



otartlng ^ites 


Cohorts . * ^ 


\ 


\11 


1^ 

Low 
Income 


All 


2_ • 

Low 
Income 


All 


Low 
Income 


All 


Low 
Income 


All 


. Low 
* Income 


All 


O 

Low 
Income 


Total 
Low 
All . Income 


I^sebud Cap 






36 


34 


73 


64 


76 


68 


55 


30 


86 


42 


326 


Z JO 


Flippin 






25 


23 


^ 24 


24 


35 


32 


52 


52 


39 


34 


175 


165 


Cherokee 










100 


84 


118 


105 


100 


81 


97 


72 


415 


3^42 


Grand Rapids 


187 


157 


232 


•?.21 


248 


242 


267 


265 


8 


0 






942 


885 


Racine 


90 


60 


122 


96 


141 


109 


134 


93 


161 


146 


201* 


50 


849 


554 


West Iron Co. 


53 


31 


60 


46 


40 


36 


44 


39 


' 24 


23 


28 


21 


249 


196 


Flint 






130 


82 


167 


95 


91 


69 


100 


89 


115 


89 


603 


424 


Todd County 




1 


117 


90 


135 


118 


169 


153 


138 


130 


145 


120 


705 


612 


Chicago 






32 


12 


36 


15 


45 


20 


24 


10 






137 


57 


Washington 






46 


45 


82 


80 


73 


65 


95 


89 


101 


70 


397 


349 


Brooklyn 


44 


37 


56 


56 


54 


54 


54 


54 


90 


89 


87 


78 


385 


368 


Providence 


1 


1 


117 


117 


224 


218 


186 


182 


185 


185 


203 


189. 


916 


892 


£. St. Louis 










116 


80 


237 


227 


280 


276 


263 


. 257 


896 


840 


Totals 


376 


287 


973 


822 


1440 


1219 • 


1529 


1372 


1312 


1200 


1365 


1022 


6995 


5922 



TOTAL 



*Mo8t of these must be Low Income. This is a miscoding that has to be corrected yet, 
**These small groups arise from placing retentions and skips with grademates. 



Table 2.3 



N for All and Lov-Income Ist-Starting Students in the- MAX-ll-^INS Analysis 
by Sife and Cohort. Retentions with Grademates* 



Ist-Startihg 


Cohorts 




All 


1 

Low 
Income 


All 


Low 
Income 


All 


2 

Low ■ 
Income 


All 


•4 

■ Low 
Income 


All 


5_ 

Low • 
Income 


AIL 


6 , 
Low 
Income 


Total 
Low 

All Income ' 


Rosebud Cap 


30 


29 


86 


70 


















.116 


99 


Dimmltt 






173 


86 


137 


66 


142 


82 


172 


97 


"190 


133 


814 


464 


Flip pin 






26 


23 


1* 


1 














27 


24 


Smitnville 


28 


28 


80 


59 


76 


66 


85 


70 


81 

^ 

99 


50 


90 


67 


440 


340 


Tupelo 


86 


83 


109 


99 


95 


91 


117 


108 ■ 


86 


105 


96 


611 


563 


Cherokee 










110 


103 


16 


16 










126 


119 


Racitie 


68 


35 


10 


9 


















78 


-44 


E* Las Vegas 


115 


89 


102 


86 


105 


95 


110 


93 


128 


' loi 


112 


96 


672 


560 


Uvalde 


115 


88 


147 


116 


161 


133 


135 


111 


126 


124 


119 


118 • 


803 


690 


Todd County 


92 


85 


133 


122 


20 


18 


3 


3 • 


1 


1 






249 


229 


Dayton 


266 


194 


221 


166 


252 


181 


243 


181 


256 


180 


302 


' 226 


1540 


1128 


Chicago 






30 


13 


2 


2 














32 


15 


Williamsburg 






130 


130 


143 


143 


102 


100 


145 


142 


156 


122 


676 


637 


providence 






131 


131 


8 


•\ 
O 














13? 


139 


E. Sc, Louis 


202 


195 


147 


140 


178 


175 


5 


4 










532 


514 


Totals 


1002 


826 


1525 


1250 


1288 


1082 


958 


768 


1008 


781 


L074 


858 


6855 


5565 



TOTAL All K and Isfc Starting - 13,847 



*These small grot'ps arise from placing retentions and skips with gr-ademates. 
01 . 
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of the N's for All and Low Income show that 80 percent of the students in 

our Follow Through classrooms come frcra Low-Income fatnilics as defined by 

the federal poverty guidelines* 

The racial cdrapositlon of the sites is as follows: 

Mostly Native American: Todd County, S. Dakota 

Rosebud Tribe, S. Dakota 



Mexican American: 

Spanish: 
Mostly Black: 



Mixed White and Black: 



Mostly White: 



Cherokee, N. Carolina 

Dimmitt, Texas (50 percent) 
Uvalde> Texas (90 percent) 

E. Las Vegas, N. Mexico 

Tupelo, Mississippi 

Grand Rapids, Michigan 

Flint, Michigan 

Dayton, Ohio 

Washington, D* C. 

Williamsburg County, S. Carolina 

Brooklyn, N. York 

E. St. Louis, Illinois 

Racine, Wisconsin 
Chicago, Illinois 
ProvidencepTlhode Island • 

Flippin, Arkansas 
Smithville, Tennessee 
West Iron County, Michigan 



The N's in Table 2.2 and 2.3 are usually larger than the N's 
presented in later analyses because some children do not have all tests 
each year. 



INTERPRETATION OF DATA PRESENTATIONS 
When presenting data which are referenced to a national norm 
population, we will provide percentiles, mean standard scores and their 
standard deviations, and mean , grades-equivalent scores, ihe following 
abbreviations are used: 



5. 
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N.. « NiAiber of cases J . ,\ 

Z-tHe « Percentile for mean standard score * 
S. S. Mean standard score 

S. D. - Standard deviation of the standard scbres 
G, E. « Mean grade-equivalent score 

In referring to tests and tes*" variables the following abbreviatfons 

4 

Will be used in the tables: 

WRAT « Wide Range Achievement Test X , - 

R » Reading (decoding) 
A a Arithmetic 
S « Spelling 



M AT ^ Metropolitan'" Achievement Test 

TR » Total Reading 

WK = Word Knowledge 
WA = Word Analysis 
R « Reading (corapreheMion) 



TM » Total Math 



7 



MCom « Math 'Computation ^- ^ 

MCon - }izch Concepts 
MPS «> Math Problem Splving 

Sp. « Spelling 

• Lag* ° Language ^ ^ 

In graphing honn-ref erenced comparisons we will present the data in 
percei\tiles on a one-fourth standard deviation scale . Since percentilt-ti 
are easily interpreted with 50 being the median for t^ie norm group, they 
are preferred in presenting results to most groups. However, percentile 
changes near 50 tend to overemphasize differences, while those near the 
extremes tend to underemphasize differences. This problem can be 
corrected for by plotting percentiles on a standard--deviAtion based scale. 
Furthermore, wht.i this is done, the sizes of differences shown can be 
directly interpreted for their educational significance. Generally, a 
one-fourth standard deviation gain has been accepted as a minimuxd criterion 
for educational significance.^*^ Table 2.4 illustrates a norm-referenced 

Table 2 A 
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Hypothetical WRAT Reading Percentiles by Grade Level 
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3 




comparison for hypothetical WRAT reading data. The percentile for the 
mean standaril score at pretest is 16. This is one standard deviation 
below the national mean. This pretest average is then taken as a 
reference point for improvement cr. the national norms at the end of 
f IrstN^)^ ^second (2), and third (3) grades. The size 
of the improvement- is shown by the length of the arrow in one-fourth 
standard deviation units. 

The reader should carefully note the differences in the nature of 
standard scores and percentiles used for the WRAT and the M.'^T For the 
WRAT, standard scores are defined for a given age group and have a mean 
of 100 and a standard deviation of 15/ We have computed our summary 
statistics from these standard scores (as well as' grade equivalents) 
and then rcr erted the mean standard score tc a percentile. This ^ 
conversion assumes a normal distribution "of the standard scores in the 
norm group (a reasonable assumption). 

The MAT uses expanded standard scores to provide a basis for 
comparison of scores on the same measure from different test levels and 
different forms. The mean standard score increases an average of 10 
points a year and has a standard deviation of 10. These scores make 
it possible to directly compare standard scores from the Primary 1 test 
with those from the Elementary level test, etc. To get percentiles, 
raw scores wefe converted to standard"^s^ores . Then statistics were 
computed on the standard scores and the mean r^tandard scores were converted 
to percentiles. 

Tlie Rrade-cquivaient scores reported are the means of the in- 
dividual ^^rade-equi vai nt scores unless otherwise specified. Thes^ 
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means will tend to be higher than the grade equivalents of the median 
score (50th percentile). For ex^jple, on MAT Total Reading, the gr^de 
equivalent of ,th^ median is 3*5 at the end of third grade rather than 
3.8. 

Summary 

In this chapter we have described the procedures used to collect 
the data on our students. We have also detailed tbe methods of data 
analysis used to enable us to focus the analysis on full-terp Follow 
Through children from Xow- Income families. We have taken special cafe 
to note two ways of dealing with retentions and skips, placing them with 
agemates and grademates. For our primary presentations they will be 
placed with .demates. We have defined the basis for three kinds of 
"gains" analyses, the MAX-N GAINS, the BETWEEN-YEARS GAINS, and the 
PRE~POST GAIHS. Because of sample- limitations at this time, the 
PRE~POST GAINS analysis will be useful primarily with Ist-starting 
sites. 

The composition of the students In the "gains" analysis was de- 
tailed by site, by cohort, by income status, and ethnic group. 

Finallv, a guide to interpretation of the data presentations was 
provided. We are now ready to get into the actual findings. 



CHAPTER 3 

RESULTS FOR KINDERGARTEN-STARTING SITES 
AND FIRST-GRADE-STARTING SITES 
In this chapter we will address ourselves to Che overal! evidence 
pertaining to objectives 1 and 2, namely: 

1. To demonstrate that economically disadvantaged children can be 
taugh't so that they will show a rate of progress typical for national 
norm groups in reading andjnath for grades. K to 3. ^ 

2. To«.<llemonstrate economically disadvantaged childrer can be 
taught so that they show a mean IQ gain from entry grade through third 
grade, eV<in though the usual expectancy is for a loss on IQ tests • 

ACHIEVEMEnI TEST DATA FROM THE MAX-N GAINS ANALYSIS 
The najor presentation of findings will use the data from the 
MAX-N GAINS analysis as described in the last Chapter, Later in this 
chapter we will present a comparison of the various gains analyses to 
show that the MAX-N GAIN analysis provides a conservative picture' of^ 
results while utll^izing the largest data base, 

K-S tarting. Low-Income Students 
A major goal of the E-B model has been to get Low-Income Follow 
Through students at a par with na^tional norms by the end of third grade. 
Table 3.1 gives one overall indication of the degree to which we have 
been successful in meeting this objective. Using the entry level 
performances on the WRAT in Reading, Arithmetic, and Spelling as the 
baselines, subtotal gains are apparent against the national norm 
group on all measures. The length of the arrows expresses the magnitude 
of gailis against the norms in quarter standard deviation units. The 



Table 3.1 here 
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Table 3-1 



Norm-Referenced Cains on the WRAT and MAT from Kindergarten througb 
Third Grade for K-^Starting, Low-Tncome, E-B Follow Through Students' 
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2A35 1988 2676 1989 2520 197A 

13 33 19 5A 8 49 
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. 18 5.35 .19 3.99 -.09 3.79^ j 
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children have learned decoding skills well (WRAT Reading) and fall 
above or very near the 50th percentile (national median) or all measures 
except MAT Total Reading, where they fall short by one^uarter standard 
deviation (AOth percentile) • ^ 

A better perspective of the meaning o.f these findings is gained 
by comparing the MAT , performances of the E-B students as shown in Table 
3.1 with that of the students in the Follow Thyough National Evaluation of 
Cohort II. The Cohort II study compared Follow 

Through (FT) students starting kindergarten in 1970 with Non-Follow 
Through (NFT) comparison groups. The Follow Through students were^ 
taken from 13 different sponsors including E-B. Table 3.2 shows the 
E-B MAT data in comparison to the overall end of third grade findings 
for FT and NFT groups in Cohort Il.j The performance of the E-B 
students is higher in each case. In compariison to 13 Follow Througfi 
sponsors, our MAT scores are one-Syalf st;andard deviation higher in Reading, 
k tull standard deviation higher in Math , and a half standard deviation 
higher in Spelling. We lack the FT comparison figure for Language. 

The Non-Follow Through control group has been found by ABT to be 
superior to the Follow Through group on most measures of socioeconomic 
status. For example,, their average income was $5964 versus $4733 for 
Follow Through,, there were fewer females as household heads (34% versus 
43%) and more Whites (34%, versus 19%). The common experience is that the 
most needy were designated for Follow Through and the control groups 
were selected from what was left. Even given these advantages, it can 
be seerr in Table 3.2 that the E~B children outperform the NFT group 
on every measure. The performances E-B students on MAT Math and Language 
are particularly • impressive . 



Table 3.2 here 
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Table 3.2 

. MAT scores at the end of third grade for K-starting Low-Income 
students in the E-B program, in programs sponsored by 13 Follow Through 
Sponsors (FT), and in the 'control groups for the 13 Follow Through 
Spons.brs (NFT) * ]?T and NFT data are from the National Evaluation of 
students entering kindergarten in 1970,^ 
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We have placed a nUmb'er of types of grade-equivalent scores at 
the bottom of Table 3.2 to illustrate some of the problems^n their 
interpretation. The Mean G. E . which is what we will typifcally report, 
will usually give a higher score than that obtained by finding the 
grade equivalent for a mean standard score ( Mean S. S. -4 G. E .). 
This is because there are more extreme grade equivalents above the 
median than below the median. Also note that the grade equivalent 
score for the 50th percentile ( 50%-tile G. £ .) varies groatly from 
scale to scale on the MAT' (as much as a half a grade level) Only by 
comparing the mean standard score grade equivalent with Che 50th 
percentile-^grade-equivalent can one fairly judf^e how good a performance 
is relative to the norms when using grade equivalents. The direct 
plotting of 4^e percentile values for mean standard scores, as we will 
usually do in this report, circumvents these problems. 

Outcgmes by Grade Level 

Table 3.3 shows the WRAT Reading and Arithmetic performances for 
E-B Low-Income students by each grade level. Note that the N's for 
some grade levels exceed 5000. The pretesting N i3 low because the 
first cohort was not tested at all on pretest and others were only 
partially tested. The third grade N is down because two cohorts have 
not yet reached third grade. The by--grade analysis for reading sKows 
that gains relative to the noim groupwere made in kindergarten and 
first grade on WRAT Reading and then maintained thereafter. The data 
for arithmetic show that there is a major gain from kindergarten in- 
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structlon which is then partly lost, but a better than average performance 
is maintained. The analysis by cohorts (to be presented later) shows 
a subtantial improvement by later cohorts. 



Table 3-3 



Table 3. A presents the VAT Reading, Math, and Spelling scores 
by grade level. The MAT Total Reading scores show a decline of three- 
fourths of a standard deviation from the end of first grade to the 
end of third grade' in comparison to the norm group. Such a decline 
is not found for Math or Spelling. 



Table 3. A 



Table 3.5 shows that the decline in standard score* for MAT Reading is 
consistent for all three sub-scores; Word Knowledge, Word Analysis, and 
Reading. An attempt to interpret this finding will be presented in the 
Discussion Chapter. 

Table 3.5 

The analysis of MAT Math subtests show inconsistent minor variations 
with grade level. The trend is toward some continued improvement against 
the norm group from second to third grade. Our students do especially 
well in math computation at the end of third grade. 

-Table 3.6 
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Table 3.3 

Norm-Referenced Gains on uhe WR/\T by 
Grade Level for K-Starting, Low-Income, 
E-B Follow Through Students 
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Table 3.4 



Norm-Referenced Gains *6n the MAT by Grade Level for 
K-"Starting» Low--lncome^i;:i;-B Follow Through Students 
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'Horm-Referenced Gains on MAT Reading Subscales by Grade 
for K-Starting, Low-Income, E-B Follow Through Students 
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Table 3-6 

Nor5i'-R:?ferenccd Gains on HAT Math Subscales by Grade 
for 'K-Scarting, Low*-Incorae, E-B Follow Through Scudencs 
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Ist-Starttng Low-Income Students 
In Ist-starting sites, there is one less year to teach the children. 



Also several of our Ist-starting sites in the Deep South and Southwest 

/ 

have our lowest performing children on entry in terms of basic language 
- competencies • Even given these limitations, the data in Table 5,7 show 
that Low-Income students from Ist-starting site3 are at or ai^f/ve the naticnal 
norms at the end ot third grade on WRAT Reading, MAT Math, and HAT 
Language* On both the WRAT and the MAT they fall one-quarter standard deviation^ 
short of the norm on Spelling, and one-half standard deviation short of MAT 
Reading, 



Table 3.7 



Outcomes by Grade Level 




Table 3.8 



ITie findings over grade levels for the HAT show an improvement against the 
norms from second to third grade in Math, and losses in Reading and Spelling- 
Table 3-9 
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Tabic 3.7 



Norffi-Referenced Cains on che WHAT and MAT 
chrouRh Third Grade for Ist-Scarcing, Low-Income, 
E-B Follow Through Students 
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Table 3.8 ^ , 

Nona-Referenced Gains on the WRAT by Grade Level 
for Ist-Starting, l^- Income, E-B Follow Through Students 



Reading 



Arithmetic 



Spelling 



-5H- 



:RIC 



4 
























PRE 


1 








1 








1 






POST 




1 












3 


1 


J 


^ 3 


H* 


241? 






3629 






4^i61 


36 3C 


2421 5173 


4646 


3609 


%til8 




4 i 


6 J 


h 7 


26 


4': 


35 


45 


20 37 


36 


30 


S. S. 


84.9 






106,3 


90.4 


9 7.9 


94. 7 


97.9 


87.4 95.1 


94. 5 


95.9 


S.O. 


.14.-4 


18.2 


:s.4 


19..' 


13.6 


12.1 


9.0 


9.8 


15.1 12.9 


13.1 


15.1 


ir b. t. 


.51 


2.02 


3.54 


A. 66 


. 7 7 


1.9: 


2.69 


3. 78 


.64 1.75 


2.69 


3.59 
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Table 3.9 

Norm-Referenced Gainc on the MAT by Grade Level 
for Is t-S carting, Low-Income, E-B Follow Through Students 

Total Reading Total Math Spelling 



84 : 1 ^ ^ 




POST 


1 




3 


1 


2 


3 


2 


3 


N 


1266 


1681 


19 31 


1269 


1676 


1926 


1654 


1863 


%-tile 


41 


43 


30 


40 


35 


48 


48 


39 


S. S. 


)7.i 


50.=? 


55.0 


38.0 


54.3 


69.9 


56.2 


60.5 


S.O. 


8.6 


7.4 


8.9 


10.4 


3.6 


11.1 


10.2 


10.8 


G. £. 


1.81 


2.60 


2.99 


1 .68 


2.58 


3.84 


2.96 


3.41 
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Analysis of MAT Reading and Math subscales over grade levels (Tables 
3»10 and 3.11) show a superior performance of our Ist-starting students 
on Word Analysis in first grade (relative to the other reading measures) » 
and a superior performance on Math Computation in third grade. These 
findings are consistent with known program strengths. 

Table 3.10 and 3.11 



Comparison of Results for K-Sites and Ist-Sites 
Table 3.12 compares the mean standard scores on MAT and WRAT 
tests for K-starting and Ist-starting sites. The entry points^n the 
UTIAT are generally comparable ov a little lower for K-starting children. 
By the end of third grade, there is a clear and highly significant 
advantage for the K-starting students. On WRAT Reading the advantage 
is more than one-half standard deviation. On WRAT Arithmetic and 
Spelling, the advantage is one-quarter standard deviation. On the MAT, 
the advantage is about one-quarter standard deviation for each measure 
except Language (which is not shown n Table 3.12). There is no 
difference on the Language subtest. This test measures grammatical 
skills which are taught to all our children in second and third grades. 



Table 3.12 



Page Ula 



o 

cr 
o 



C3 



o 

cr 
CD 

<o 

cr 
o 

</> 



o 
a. 



84 
77 
69 
60 

50 

40 
31 
23 



11 

7 
4 



Table 3.10 

Norm-Referenced Gains^on MAT Reading Subscales by Grade 
Level for Ist-Starting, Low-Income, E-B Follow Through Students 



Word 
Knowledge 



Word 
Analysis 
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1 




1 
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Reading 



POST 


1 


2 


3 


1 


2 


1 


2 


3 


N 


1268 


1687 


1933 


1270 


1686 


1271 


1687 


1943 


%-tii8 


44 


* 

38 


30 


62 


46 


39 


40 


32 


S. S. 


-39.0 


52.2 


56.9 


39.7 


51 .0 


37.4 


51.1 


55.6 


S.O. 


10.0 


7.5 


8.3 


6.8 


8.7 


9.0 


8.4 


10.1 


G.E. 


1.87 


2.65 


3.08 


1.91 


2.83 . 


1.74 


2.58 


3.00 



Table 3.11 
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Norm-Referenced Gains on MAT Hath for 
Ist-Starting, Low-Income, E-B Follow Throu^ Students 
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POST 


2 


3 


1 


2 


3 . 


2 


3 


N° 


1683 


1938 


1269 


1685 


1934 


1684 


1935 


%-tile 


Al 


68 


AO 


28 


42 


37 


40 


S. S. 


52.2 


69.8 


38.0 


51.2 


64.1 


53.3 


65.0 


S.D. 


9.0 


11.7 


10.4 


9.7 


10.9 


9.9 


11.4 


G.E. 


2.73 


4.30 


1.68 


2.57 


3.71 


2.68 


3.64 
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Table 3.12 ,^ 

A Comparison of Norm-Referenced Gains on the 
WRAT and MAT from K-Starting and Ist-Starting 
Sites for Low-Income E-B Follow Through Students 



R 

KS-IS 



- WRAT- 
A 

KS-IS 



S 

KS-IS 



TR 
KS-IS 



- MAT- 
TM 

KS-IS 



Sp 

KS-IS 



i; 



11 



4 






















PRE 


K 


1 


K 


1 


K 


1 


K 


1 


K 


1 


POST 


3 


3 


3 


3 


3 


3 


3 


3 


3. 


3 . 


N 


1988 


3629 


1989 


3630 


1974 


3609 


1863 


1943 


1804 


1926 


%-tile ^ 


83 


67 


54 


45 


49 


39 


40 


32 


52 


48 


S. S. 


114.5 


106:5 


101.2 


97.9 


99.7 


95.9 


58.0 


55.1 


71.1 


69.9 


S.D. 


20.9 


19.2 


8.6 


9.8 


13.9 


15.1 


10.3 


10.1 


11.9 


11.1 


6.E. 


5.35 


4.66 


3.99 


3.78 


3.79 


3.59 


3.31 


3.00 


3.98 


3.84 



K 1 

3 3 

1824 1863 

48 39 

62.8 60.5 

11.3 10.8 



3.^8 3.41 
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Effects of Placin^^ Retentions with Agemates 
Versus Grademates 

In the data presented so far, we have placed children who have been 
retained one grade or more with their grademates. We have also analyzed 
the data keeping retentions with their agemates (their starting cohort) • 
For a norm-referenced comparison, it makes sense to keep retentions with 
their grademates, since norms are built with this procedure. For 
K-s tar ting Low- Income students, retentions were 6.2 percent. For Ist^ 
starting Low-Income students, retentions were 11 percent. In some. of our 
Spanish speaking sites it was customary to retain over 60 perpl^nt of the 
students in first grade prior to Follow Through. We coul/ find^no national 
statistics on retentions. However, judging from percent of children, above 
the modal age of fourth graders, we would estimate retentions to fall between 
8 and 12 percent nationally. In general, ps sponsor, we have discouraged 
retentions until this year. We now encourage keeping the children back 
until they complete three full years of the Distar programs. 

Table 3,13 shows that the effect of placing retentions with 
grademates is to increase the total number of children in the analysis at 
the end of third grade, but not to appreciably change the findings. 
On the WRAT, there is at most the increase of one standard score unit 
(one-fifteenth standard deviation unit) when retentions are placed with 
grademates. This is a negligible effect. On the MAT the changes are 
variables. The. mean standard score for Language goes down for K-starting 
students, Reading and Math means do not change for K-starting students, 
but show a small increase for Ist-starting students. 



Table 3.13 
_ _ — — ^— 

< u 
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Table 3.13 

Effects of Plrcing Retentions with Agemates Versus 
Grademates (Low-Income students only)* 



K-Starting 



Variable 


With "Grademates 


With Agemates 


Post 3rd Grade 


Mean S.S. 


S.D. 


N 


Mean S.S. 


S.D. 


N 


WRAT 














Reading 

Arithmetic 

Spelling. 


114.0 
101.2 
99.3 


20.5 
8.6 
13.3 


1911 
1912 
1897 


113.0 
100.9 
98.9 


20.2 
8.6 
13.2 


1866 
1867 
1851 


MAT 














Total Reading 
Total Math 
Spelling 
Language 


58.0 
71.0 
62.8 

^69.9 


10.3 
12.0 
11.2 
13.2 


1784 
1727 
1708 
1775 


58.0 
70.8 
62.4 
71.3 


10.5 
12.8 
11.5 
13.0 


1593 
1547 
1532 
1470 


WRAT 




Ist-Starting 








Reading 

Arithmetic 

Spelling 


105.3 
97.5 
95.0 


18.4 
9.4 
13.9 


3460 
3461 
3440 


104.8 
96.8 
94.7 


1 

18.0 
9.7 
13.8 


3224 
3230 
3216 


MAT 














Total Reading 
Total Math 
Spelling . 
Language 


55.0 
69 .'8 
60.4 
70.6 


8.9 
11.1 
10.8 
12.1 


1883 
1878 
1816 
1893 


54.0 
67.7 
59.5 
70.1 


8.9 
12.5 
11.1 
12.3 


1525 
1519 
1466 
1378 



*These N's for students with Grademates are slightly lower than those in Tables 
3.1 and 3.7 which have similar data. Thin is because the effects of retention 
placement comparisons were made prior to a final data correction which brought 
Head Start Planned Variation children into the sammary analysis, and in one site 
children grouped as "unknown** were identified as Low-Income and moved into that 
analysis. Thus, the differences reflect a conscious effort to insure tha^ a 
fair comparison is being presented. 





One would expect the 20 percent Non-Low- Income students in our 
Follow Through classes would do better than their Low- Income peers. 
Table 3.1A shows that when Non-Low-Income students are added to the 
analysis, the means go up on nearly all variables one to two standard 



comes are not surprising. An estimate of the mean standard score for 
Non-Low-Incorae students can be crudely obtained by multiplying the 
differences shown by five and adding the result to the mean for Low-Income 
students . 



in the "gains" file. This is the data base we have been looking at so 
far (the MAX-N GAINS). It is fair to question whether this data base, 
even with the large N's, truly reflects student gains within the program. 
To answer this question, two additional types of analyses were made. 
First, we analyzed BETWEEN-YEAR GAINS for all Low- Income students in 
the "gains" fi'.e. Second, we examined the outcome for students on 
whom we had both entry and exit tests (PRE-POST GAINS). For K-starting 
sites, this N is very small for three reasons: (1) early cohorts were 
not tested at entry, especially where SRI was testing, (2) two later cohorts 



have not yet finished the program, and (3) standard score norms^ on t;he 



score units. 



Standard deviations also increase slightly. These out 



Table 3.1A 




( u 
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Table 3.1 A 

Effects of Adding Non-Low-Income Students 
(Retention with grademates) 



K-Startlng 



Variable 


Low- Income 




All 


* 


Post 3rd Grade 


Mean S.S. 


S.D. 


N 








WRAT 














Reading 


114.5 


20.9 


1988 


115.5 


21.1 


2308 


Arithmetic 


101.2 


8.6 


1989 


101.7 


8.7 


2311 


Spelling 


99.7 


13.9 


1974 


100.3 


14.0 


2296 


MAT 














Total Reading 


58.0 


10.3 


1863 


58.8 


10.7 


2127 


Total Math 


71.1 


11.9 


1804 


71.8 


12.0 


2063 


Spelling 


62.8 


]»1.3 


1824 


63.2 


11.2 


2017 


Language 


69.9 


13.2 


1854 


71^0 


13.6 


2118 


WRAT 


Ist-Starting 


r 






Reading 


106.5 


19.2 


3629 


107.6 


19.5 


4432 


Arithmetic 


97,9 


9.8 


3630 


98.6 


9.9 


4431 


Spelling 


95.9 


15.1 


3609 


97.1 


15.7 


4406 


MAT 














Total Reading 


55.6 


10.1 


1943 


56.2 


9.8 


2358 


Total Math 


69.9 


11.1 


1926 


71.2 


n.5 


.2354 


n Spelling 


60.5 


10.8 


1863 


• 6l!3 


11.0 


2285 


Language 


70.7 


12.2 


1941 


72.1 


12.8 


237^ • 
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WRAT3 are lacking for pre-K children under 5 years of age. These three 
factors combine to give a low N on WRAT standaSrd scores for K-starting 
PRE-POST GAINS (N » 169 to 178). , ' 

Table 3.15 shows the results for the three types of analysis for 
K-starting students on WRaT Reading and Arithmetic. A very interesting 
effect is^ present in the BETWEEN-YEARS GAIH data. The Post-score group 
for a lower grade level always has a higher mean than the Pre-score 
group for a higher grade level. For example, post-kindergarten 
mean on Reading for 1,287 students was 110.5. The pre-first mean for 
3,766 students was 107.4. This effect is flbe to the fact that there are 
relatively more students in lower-grade-level groups from later-starting 
cohorts. Later cohorts do progressively better. 



Table/" 3.15 



When the three kinds of analyses are compared, the following 

conclusion is apparent: The MAX-N GAIN analysis conservatively 

represents the true gains mad^ by Low-Incoroe Follow Through students in 

« 

t he E-B program . If we add together the BETWEEN-YEAR GAINS, and add 
this sum to the entry score, the gain would be much greater than that 
shown in thi\MAX-N GAIN analysis. The PRE-POST GAIN also shows a 
slightly larger gain than the MAX-N GAIN. , 

Since the MAT was not given at entry, and not used at all until 
1972, a similar analysis' cannot be made for MAT scores. However, for 
the 375 students in thj^ PRE-POST GAIN analysis, the mean post-thirS- 
grade score was 59.6 on MAT Total Reading. This score is 1.6 standard 
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Table 3.15 

Comparison of MAX-N GAIN Analysis EffectP with those 
for BETWEEN-YEAR GAIN and- PRE-POST GAD', Analyses on 
K-Starting Low-Income Students^ 



WRAT Reading 





MAX-N 


gain' 


I 


BETWEEN-YEAR GAIN^ 


PRE-?OST GAIn"^ 




Mean S.S. 


S.D. 


N 


Mean S.5. N 


'Mean S.S. 


S.D, 


N . 


Pre K 
Post K 
Po&t 1 
Post 2 
Post 3 


86.1 
107.5 
115.2 
112.8 
113.1 


16.0 
18.5 
20.1 
19.7^ 
20.2 


19*»8 
A621 
A378 
3023 
1866 


86.6 

' ■\ - 1287 

110.5 107. A ^ 

t 3766 

116.2 11A.5 

I • 2825 

113.1 112. A/ 

t - 1770 • / 
113.1 / 


85.4 
113.2 


u 

13.3 
20.0 


169 

i 

.169- 










/ 

/ 








WRAT Arithmetic 




MAX-N GAIN^ 


''2 

BETWEEN-YEAR GAIN 


PRE-POST GAINS^ 
















Mean S.S. 


S.D. 


N 


Mean S.S. ' N 


Meaii S.S. 


S.D. 


N 


Pre K 
Post K 
Post 1 
Post 2 
Post '3 


. 86.6 
108.2 
105.5 
99.7 
100.9 


15.7 
•lA.A 
11.2 
.9.3 
8.6 


21A1' 

A628 

A378 

301A 

1867 


87.0 

t y . I 1398 

1W.7 108.2 

t 3767 

106.2 105. A 

^ 2816 

. 8 99 . 3 

t 1775 
100.9 


86.1 
100.7 r 


16.0 
7. A 


178 

• 

178 



As with T tble 3.13, this analysis was completed prior to making a few 
final adjustments to included Planned Variation Head Stdrt kids and to 
correct the placement of a group of Low-Income kids whose income level 
had not previously *been identifier correctly. 

^2 -—"^ 
For these data, retentions are grouped with agemates. 

3 " * 

For these data, retentions are grouped with grademates. 

#» * 

• ( 
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Rcore points higher than the toean for the comparable MAX-N G/ilN group. 

For MAT Total Math» the comparison is 74,1 for the PRE-POST GAIN group 

and 71*0 for the comparable MAX-N GAIN group T These findings again 

support the conservative nature of. the MAX-N GAIN report. 

Table 3.16 presents a similar .comparison for first-starting sites^ 

All of the comparisons shown, including those for the MAT again show tha^ 

^he MAX-N GAINS present a conservative picture of the true gains. 



Table 3.16 



Note : Since we do not have available the BETWEEN-YEARS 
retentions placed with grademates, the data for^the MAX^li Analysis 
in Tables 3.15 and 3.16 is shown for retentions with agemates. As should 
be apparent from Table 3.13, the effects of these different placements of 
retentions are minor and do not alter the conclusion drawn above.^ 

Comparison of Findings for "Singles" and "Later EnterinR*' Groups 



As noteT^TrT^apter 2, we have about 4800 student records where 

i 

there is only testing at one point in time ("Singles") ^nd thus no basis 
fot computing gains. These records arise for several reasons. In the 
early years, many students were not pretested, were given a posj/^ 



^««e 50 

Table 3.16 

Cotaparlaoa of HAX-^N GAIH Analysis Effects with those 
' for BETWEEH-YEAR GAIN imd PRE--POST GAIK Analyses on 
1-st Starting Uow-Incctse Students^ 



* RRAT Reading 





BEmEEN-YEAB GAIN^ 




, PRE-POST GAIN^ 












Mean S.S. 


S.D. 




Mean S.S. 


N 


Mean S.S. 


S.D, 


H 


; ete 1 


82.1 


11,4 


1746 


82,4 




82v2 


11.4 


837 


Post 1 










1549 








98.6 


17.1 


4111 


100.9 99.2 




















3522 








; Post z 


103.7 


18,1 


4007 


104,4 104.2 










V Post 3 








r 


2987 


1 






104.8 


18.0 


3224 


105.2 




106.3 


16.1 


837 






HAT Total Reading 






Post 3/ 




8.9 


1383 ; 

\ 

i- 


56.5"^ 


8.1 


818 



WIb\T Arithmetic 



KAX-N CAIH^ 


BerWSEN-YEAR GAIH'^ 


?RE-P0ST CAIH^ 




Hem S.S. 


S.D. 






Mean S.S. 


a 


Meao S.S. 


S.D. 


N . 


Pro 1 
Post 1 


88, 3 


U.3 
11.8 


1775 
4087 


88. 
t 

100. 


7 * 
0 98.0 


1568 


89.8 


12.0 


857 


Post 2 


94.3 


8.9 


4005 




T 

94.7 94.2 


3496 






* 


Posu 3 


96.8 


9.7 


32 30 




t 

97.1 


2989 


1G0.4 


8.6 


857 






f ^. — « — ^ 

HAT Total Math 






Post 3 


69, S-' 

i 


ii.l 

1 


187^ 








72.7^ 


10.0 


816 



S)ata based, on slightly tttsalicr nmple Bite than in Tabl^ 3,7. Stu footnote to Table 3.13* 
For these data, retentions are grouped with ageaates* 
For those data, recent lona ar*» grouped vlthN^ademates . 



ERIC 



PaRe 5i 



kindergarten test and then left the program to attend parochial school. 
In Grand Rapids two whole cohoria wore pretested in the fall and not 
tested in the spring because of a lack of a functioning memorandum of 
agreement. Many students moved from the program or were migrants. Finally, 
bad testing procedures in one site early in the program led to a large 
' aumber of single records after some tests were discarded. 

The present data on students where th^re are single records does 
not differ appreciably from chat reported in Technical Report 73-2» 
December 19 73. The students tend to show lower test perfr.i;r.nn^es in 
proportion to their lower placements in the Distar programs. The saijje 
conciuBion can also be driiwn .for the 1900 'Mater entering" f?tudents 
. our file. Th^ students perform more poorly when chey have not been 
full-term -he K-B Follow Through program. This is to be expected if 
Che program important to their success. 

mF RQVEHEHT IH_ IC^ 
Th** '''Ir.s'son I ntoUi^tence Test was used as a measure of general 
lanstuaro 'onpctencv and more generalised cognitive skills. The f Indinp^ 
generar.v sh-^w ,v.i Improvement in IQ from entry to end of third grade. 
In K'-sit*'<^ ^T.iMe rhe average improvement Is 2.3 points for 

low-income 'U.:dent-. in the PRB-POST CAIN' analysis. The BETWEEN-YL/iKS 
CMVi analvsls h .ov^ a gain of 9.6 polQts during Che kindergarten year and 
small losn.-r. tnraline, 3.^ polnt:5 after that. The MA»K GAIN analysis 
shows a j?air. ^.5 points. The PRL-POST GAIK analysis utilizes the 
data from coi\oriH i and A exclusively. U apj^ears that cohorts S and t 
are showing gains that will be closer to 6 or ? points th^f 2. 

Table 3.P 
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PRE-POST GAIH 



Table 3.17 
Slosson IQ Gains, K-St.arting Students 

Retentions with Grademates 





rose J 


Gain 


S.D. 


fi 


All 106.6 


109.2 


2.55* 


15.5 


• 346 


Low Income 105 . 7 


108.0 


2.30* 


15.5 


293 












BETWEEN-YEARS GAINS 


Retentions 


withn^gemates 




Pre 

Low Income Only 


Post 


Gain 


S.D. 


N 


Pre K - Post K 103.2 


112.6 


9.40** 


14.2 


1850 . 


Pose K - Post 1 no. 2 


108.8 • 


-1.45** 


13.1 


2495 


Post 1 - Post 2 109.8 


109.4 


- .04 


12.9 


1079 


Post 2 - Post 3 110.3 


108.5 


-1.81** 


11.6 


791 


^^Significant beyond .001 level. 








^lAX'-N CAIN 

Retention 


with Grademates 






Lov Income Only 

Mean 


S.D. 


N 






Pre K 102.5 


14.6 


2642 






Post K 109.8 


15.2 


4509 






Post ] 108.1 


14.8 


3174 






Post 2 09.1 


16.6 


2068 






Post 3 106.9 


16.7 


1933 







* P < , 01 
** P < .001 



A 
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When examined more closely, it is apparent that the IQ gains for 
K-sites are quite variable. With 73 students in the PRE~POST GAINS analysi 
Cherokee allows a mean gain of 9.4 points (101.1 to 110.5). On the other 
Viand, West Iron County shows a loss of 9.8 points (111.8 to 102.0) for 
19 students In cohort 3. This group was pulled out of the program 
before they finished It. In general, gains and losses seem related to 
adequacy of program implcrentatlon. 

The IQ gains for Ist-^sites are moic subblantial. The average is 
5 to 6 f )int3 no matter which gains analysis is examine^. This gain 
is over one-third standard deviation. Examination of the data by site 
suggests the Importance of program implementation. For example, In* 
Las Vegas, New Mexico, after two years of modest gains (A.l and 3.8), a 
push on taking the kids as far as they could go, produced a mean gain 
of 13 points for 66 children. Larger ;'a1n<v aisn t^r\<\ to hf found in 
aieas where in*- parent lan^ufw/- nr»r Fnf^lish (Ros^'bu^i, JN^qi/^^' uimmitt). 



The eff»-'cts 



iru* j.^ilnB in Ist-starcing sUtes is to ring the mean 



student Iv tu {he average for rho p^^n^r^l f^Apu] r f or . 



Tm <^ r r 1 b u f 1 on^ ] I y h e s 




It is c.^ssi'Ie tor rof-av. effe-r^ to li .il^rwit^-.i \,\ 



^ r ibut Ions . 



trjf» c :f)r t ru- present darn. 



When VP ^'X.ir tn#. 




atov« yiA \*- 



" o vorv (on^i(sr^-nt with thn<?f» alread'. 



'."fits ? pT <-n? of cidi.ireo «h<^'f' yr^A*- '»**rr 



Table 3.18 



Slosson IQ Gains, Ist-Starting Students 



PRE-POST GAINS 



Pre K 



All 95.5 
Low Income 9A . 7 

*Slgnific^nt beyond .001 level. 



Retentions with Grademates 



Post 3 

lni.9 

101.0 



Gain 

6.38* 

6.31* 



S.D. 
14.5 
1A.6 



N 
1208 
996 



BETWEEN-YEAR GAINS 

Low-Income Student 

Pre 1 - Post 1 
Post 1 - Post 2 
Post 2 - Post 3 , 



Pre 



93.6 
97.7 
98.8 



Retentions with Agematea 
Post Gain S.D. 



98.6 
99.3 

99.3 



/'.96* 
1.61* 
.A8* 



11.5 
12.1 
11.9 



1592 
2533 
215A 



*Signiflcant beyond .001 level. 



MAX-N CAIN 

Lo. Income StudcuL Mean 

Pre 1 9A.1 

Post 1 97.1 

Post 2 98,6 

Post 3 99.4 



R etention with Gradematea 
S.D. 



U.5 
14.7 
15.4 
16.8 



N 

2639 
4560 
3538 
3651 
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for the students in the MAX-N GAIN analysis (retentions with grademates) . 
The analysis shows that more than 50 percent of our K-starting Low- 
Income children are above grade norms on »iach measure except WRAT Spelling 
and MAT Reading. The Ist-starting scudents show a lower percentage 
above grade norms except on MAT Math and Language where they do as well. 
The imporcartce of using the kindergarten year for instruction is as 
apparent in these data as in the data based on means. 

Table 3.19 

Figures 3.1, 3.2, and 3.3 show the dlstribut^lons by stanine groupings 
for ALL students in the MAX-N GAIN analysis tor the Pre-K and Po8t-3rd 
testings. The distributions for WRAT Reading show a remarkable shift to 
higher stanl. .s. At pre-kindergar ten 58 percent of che children fell in 
stanines 1 to 3. Post-third grade, only 8 percent fall in stanines 1 to 
3. At pre-kindergarten, only 6 percent of the children fell in stanines 
/ to V. Post-third grade, 53 percent fall in these top level sti.ninea. 
Srronp^ rJl str Ihut ional shifts are also present for Arithmetic and Spoiling, 
but thpy arc not «i3 pronounced as that for Reading. 

Figures 3.1, 3.2, 3.3 

The percent <Age of rases by stanines on the MAT tests for K-starlin^> 
children at the one flr<Jt, second, and third grades are given in 
'^iblo i '^^<^ ^,72. inir Follow Throuf^h children do better 

r.mn the norm group on Total Reading at the end of first and seconti 
grade^i, hnl somewr.at behind on the tlilrd grade test. This is 
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Pre-K Post 3 

2137 2308 




Figure 3.1 Distribution on WPAT Reading by stanines tor Follow iliroui>li 
Students (Ail) In M^U-N GAIN analysis for Pre-K an-d Post 3rd testing. 



V. 
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Pre-K 
2315 



Pose 3 
2311 




STAN'INES 



Figure 3.2 Distribution on WRAT Arllhn-.etic by stanines for Fol lov. Throug! 
Students (All) in MAX-N r.AIN nnalvsis for Pro-K and Post 3rd testing. 
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SIANINES 



Figure 3.1 Distribution on WRAT Spell ing by scanines for Follow Throuj;h 
Students (All) in M/J(-N GAIN analysis for Pre-K and Posj 3rd testing. 
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Table 3.19 
Percent of Children above Grade Norm 



J(|-Startlng 
Variable 



WRAT 



Reading 

Arithmetic 

Spelling 



MAT 



Total Reading 
Total Math 
Spelling' 
Language 



Low Income 
N Above N Total % 



1A39 
1177 
85A 



1890 
1893 
1879 



76.1 
62.2 
A5.A 



/ All 
N Above N Total % 



177A 
1A52 
1052 



2308 
2311 
2296 



76.9 
62.8 
45.8 



716 


1777 


AO. 


3 


882 


2127 


A1.5 


925 


1719 


55. 


0 


1138 


2063 


'.55.2 


866 


1697 


5] 


0 


10A6 


.017 


51.9 


933 


1770 


52 


7 


1156 


2118 


5A.6 



1st Starting 



WRAT 



Reading 


2099 


3499 


60. 


0 


2770 


4432 


62. 


5 


Arithmetic 


15n 




4 i. 


3 


. 2025 


4428 


45. 


7 


Spelling 


1090 


3481 


31. 


3 


1505 


4' 06 


34 


2 


\T 

Total Reading 


480 


1903 


23. 


2 


674 


2356 


28 


6 


Total Hath 


971 


1904 


51 


0 


1283 


2353 


54 


.5 


Spelling 


743 


1841 


40. 


4 


985 


2283 


43. 


1 


Language 


1064 


1916 


55. 


5 


1385 


2371 


58 


.4 
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consistent with findln^is reported earlier* The new information to note 



It is only in the high 'stanines (7, 8, 9) that we fall behind in frequency 
of cases. The implication is that the program produces students with good 
reading comprehension, but we lack our share of the top level kids in 
language competency. This is to be expected if disadvantages child 
are on the average weak in language competencies. Our program compensates 
for this by building a basic understanding of the language of Instruction, 
but cannot in the time available build adult-like vocabularies for some 
of these -children. The latter apparently occurs in many middle-class homes. 

On the MAT Total Math test (Table 3.21), percentage of cases by 
stanyfnes shows an improvement from fir-^t to third grade for K-starting 
children. Table 3.22 shows a distributio.-^ that is a little better than 
the norm group for Language (grammar) and a little worse for Spelling. 



is that in the low stanines (1, 2, 3) we match the norm group quite well* 



Table 3.20 



All Children from K-Sites 



MAT Toi:al Reading Percentages at a Given Stanine 



Percent at Each Stanine 



Stanine 



Post 1:^1 



Post 2nd 



Post 3rd 



Norm 

Expectation 



3 






5 
6 






17 ; 



7 

h 
9 






N « 3183 



2556 



2127 



Stanlne 



Table 3.21 
All Children from iC-S.'tes , 
MAT Total Math Percentages at a Given Stanine 



1. 
2 

3 13 
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Parcent at Each Stanine 

Norm 

Post 1st Post 2nd Post 3rd ExperJ-^tion 



9^ 21- 26 7^ 20- 

3 ) 14 / 9 3 12 ; 



23 



4 
5 
6 


22 \ 
16 ) 


55 


\ 

14 J 


52 


16-^ 

17 y 51 

18 J 


20 

17 ] 




7 
8 
9 


n 


17 


■1 


23 


8 }30 


"il 


' 23 



N = 3207 



Table 3.22 
All Children from K~Sltes 
MAT Language and Spelling Percentages at a Given Stanine 



Stanine 



4 
5 
6 



MAT Language MAT Spelling 

Norm 

Pose 3rd Post 2nd Post 3rd Expectation 



1 A 



2 

3 10 



6^ 20 P^20 . J I 22 ^7"! 23 



17-) 18-) 16--) 17->) 

20 (56 17 t 57 21 f 58 20 ( 50 

19 J 22 ) 21 ) . 17 ) 



7 11-^ 10 

8 8 y ~ 

9 5 ) 



8 y 24 o| 23 20 7^ 23 



N - 2118 N - 2539 N - 2017 



b manary 

The data presented in this Chapter show that the obiective of 
getting disadvantaged children up to national norm averages on b?sic 
academic skills was largely met, especially for K-starting sites. 
Improvement on the Slosson IQ tests was also found, especially with 
the initially lower Ist-startin^ sites. 



CHAPTER 4 



ANALYSES BY COHORT 

The dnta by cohort pennit an examination of the reliability of 
the findings over time and an excmination pf possible prog^afc Improvements 
over time. As was shown in the last chapter, the MAX-N GAINS data base 
leads to the sajne conclusions as the other more refined gains analyses 
(with a slight reduction in magnitude of effects), while providing for 
a much larger" sample size. For these reasons we will restrict analyses 
in this chapter to that data base, except for dat^. We will further 
focus the analyses ohly on Lov-Incorae students with retentions placed 
with grodemat'^a. 

READING 

K-S tar ting Sices 

Yearly gains (or losses) against the norm group are 8ho\/n for 
WRAT Reading in Table ^-,1 md In these tables, the ending 

level from the grade before is used as the base for change for the 
ne^ct grade ieveJ. » data for kindergarten show littl'* varla- 'on 
In the entry perfonjncc of the childrep, a strong gain against the 
norms for all cohorts, ind a three-fourthb stan<}ard deviation improv jment 
from cohort 2 to coh rt This improvement can be attributed largely 
to the improvement in teaching at the kindergarten level. In the v^rly 
years^ kindergarten teachers wete t prepared to believe that their 
children could be taught to read, >o they did not always **follow through" 
winh sponsor suggestions. After a number of demonstrations of what could 

-/ 

t 
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be done, and a real targeting by the sponsor on kindergarteii- level 
performance of teachers and aides, improvements were made in N^he 
nuiber of lessons taught and the quality of the teaching.' 



Table 4.1 and A. 2 



The data for cohort 1 stands apart from the rest. These data arV 
from only four sites and the N is small. However, as we shall see in \ 
'Tables 4.3 and 4.4, a similar effect is found for Ist-starting sites. \ 
One explanation for the cohort 1 effect is that the reading (^n'l a«rith- 



metic) program the first year moved faster than the publishe^^version ^ 
the following year which gave more atteution to lower performing children* 

Vhe data over cohorts for first grade, also show rather consistent 
gains i^gainst ,ne norms and that improvements over cohorts tend to be 
maintainiad. The data in Table 4.2 for second and third grades, show that 
there are tao further gains against the norm group. On the contrary, 

except for Cohort 1, there are slight losses in second grade and 

\ 

essentially ni? changes in third grade. The pattern for level of per- 
formance over \ohorts at the end of third grade is similar to that 
found at the end of kindergarten. 

Discounting the cohort 1 data, we can conclude that the improved 
teaching of reading over cohorts in kindergarten has measurable effects 
which are still apparent at the end of third grade. 
Ist-Starting Sites 

Tables 4.3 and 4,4 present the WRAT Reading data over six cohorts 
for Ist-starting sites. The tables show consistent gains against the 
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Table 4*1 

NoVmrReferenced Gains on vmAT Reading by Cohort frotn 
Kindergarten through First Grade for K-Starting, 
\ Low-Income, E-B Follow Through Students 



First Grade- 



"1 ^ 




Cohort 


1 


2 


3 


4 


5 


PRE 


K 


K 


K 


K 


K 


POST 


K 


K 


K 


K 


K 


Post N 


97 


755 


1184 


1173 


1045 


Post %-tile 


61 


4> 


66 


70 


77 


Post S. S. 


104 


99 


106 


108 


111 


Post S.D. 


18.9 


17.9 


19.6 


18.5 


19.0 


PR?r^t 6. E. 


1.24 


1.04 


1.32 


1.39 


1.49 



73 
109 



K-1 K-1 . K-1 K-1 K-1 K-1 
281 758 ^135 988 889 759 



79 



79 82 



112 112 114 



88 87 87 
1:8- 117 117 



96 



,1.1 20.6 19.5 21.2 19.9 20.4 
2174 2.73 2.86 '3.08 3.05 3.00 
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Tabled 4.2 



Norm-Referenced Gains (Losses) on VfRAT Reading by 
Cohort from Post-First Grade to Post-Third Grade for 
.K-Starting, tow-Income, E-B Follow Through Students 



OS 

o 

CO 



CO 

CO 



CO 

c: 

CO 
CO 



LA- 



89 



Second Grade 



Third Grade 



84 

7/- 
69 
60 
50 

40 
31 

23 
16 



11- 

7- 
4 



Post 



Cohort 


1 


2 


3 


A 


5 




1 


2 


3 


4 


POST 


1-2 


1-2 


1-2 


1-2 


1-2 




2-3 


2-3 


2-3 


2-3 


N 


234 


655 


797 


854 


758 




197 


409 


664 


718 


%-tile 


86 


75 


81 


84 


84 




86 


79 


82 


86 


S. S. 


116 


110 


113 


115 


115 




116" 


112 


114 


116 


S.D. 


18.5 


18.3 


19.6 


21.0 


20.6 




18.4 


18.3 


20.9 


22.8 


G.E. 


4.44 


3.89 ' 


4^:22 


4.26 


4.36 


QQ 


5.35 


5.06 


5.35 


5.51 
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/ 

norms for first and second grades, which are maintained in third grade. 

The gain during the first year of instruction is particularly dramatic. 

However^ there is no dramatic trend for ar improvement over cohorts as 
i — "~ — — — — _ — 

Tables 4.3 and 4. A 

found for K-startir.g sites. In general, we were working with experienced 
teachers who were very* hard workers and showed good implementation of 
the reading program from the start. 

The high performance of cohort 1 at the end of third grade Is a 
little perplexing. This cohort also tends to do better in spelling, 
and math. No MAT data are available for this cohort since they graduated 
before we began MAT testings* A careful examination of the data by 
site shows Cue possible contributions of several factors. First, we 
know in E. St. Louis that the first gro^p was taught by th^ best 
teachers in the district and the children were strongly taught all the 
way through. They ended up -with a mean WRAT Reading standard score of 
129 (N « 166) for cohort 1. The m^an for cohort 2 was **only" 115 
(N « 122). Second, wemote that in five other sites there are similar 
strong gains against the norm group from post-second to post-th^rd 
grades for .cohort 1 children, whilfe the trend is for a loss against 
the norms the following year for cohort 2 children. We suspect that the 
initial excitement of a new program, combined with the field-test version 
of the reading program that moved more quickly than the revision must 
account for part of this unusual performance. Third, when tha site 
changes are considered, we note that a high performing site that 
started in K and 1st simultaneously is present in cohort 1, but not 

■I. V V 



PaKe 68 

Table 4.3 

Norm-Referenced Gains on WRAT Reading by Cohort from 
Pre-First Grade\hrough Second Grade for Ist-Starting, 
Low-Income, E-B Follow through Students 



CO 

o 

CO 



€9 



' c: 



CD 
IT 

CIS 

c: 
o 



. c: 
o 



84 



77 



69 
60 
50 

40 
31 
23 



16 



•First Grade 



Second Grade 



' 6 
1 



Cohort 12 3 

PRE I 11 

POST 

Post N 

Post %-ti!8 53 45 47 50 53 53 

Post S. S. 101 98 99 100 101 101 

Post S. D. 20.0 17.7 17.9 18.9 16.6 18.1 



111111 
721 1208 1009 746 713 763 



ERlCist G. E. 



2.11 1.89 2.01 2.01 2.07 2.11 

1 



1-2 1-2 1-2 1-2 ' 1-2 1-2 

775 1163 977/ 604 608 538 

63 66 61 58 63 63 

105 106 104 103 105 105 

18.7 19.1 18.4 18.0 17.0 18.2 

3.56 3.68 3,41, 3.45 3.59 3.53 

Hi 



Table 4.4 



03 
O 

cs 
•S 

a> 
• 

OS 

c: 



o 

f 

e: 
«Q 
o 

CO 

c: 
o 



Norm-Referenced Gains (or Losses) on WRAT Reading by 
Cohort from Post-Second Grade to Post-Third Grade for 
Ist-Starting, Low-Income, E-B Follow Through Students 

Third Grade' 











> • 
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Cohort 
. f'OST 
Post N 
PosV%-tile 
Po^t Is. 
PiJSt S.O. 

<^ t fi F 



1 


2 


3 


4 


5 


2-3 


2-3 


2-3 


2-3 


2-3 


678 


1012 


853 


602 


481 


82 


63 


■63 


58 


63 


114 


105 


105 


103 


105 


22.5 


18.0 


19.6 


18.6 


16.3 


5.48 


4.50 


4.50 


4.43 


4.A4 



cohort 2; while a low-performing site was new to Follow Through in 
Cohort 2. Ihv^se three effects combine to produce an effect that is one- 
half standaid deviation in magnitude. Note that most of the effect 
is accounted for by program and teacher variables. It is perhaps also 
important to note that cohort 1 showed significantly highe'«- IQ gains 
oa the SIT from the end of first grade to the end of third grade than 
the average of the other cohorts (96,9 to 102,0 versus 97.1 to 99.0 
with N's <ff approximately 600 and 3000 for the two groups), (A similar 
comparison from the start of first grade is not possible because cohort 
1 was not pretested.) 
MAT 

The analysis over cohorts for MAT Reading scores is not very 
productive, since the MAT was not added to the testing until 1972, 
We can note, however, that for K-s tar ting sites in cohorts 2 and 3, 
there is an increase on MAT Total Reading standard scores at the end of 
third grade from 56.5 (N A87) to 58.8 (N = 656), and then a slight 
drop for cohort 4 to 58.3 (N = 720). The lesser gain from 56\5 to 
58.3 is significant beyond the *001 level (CR = 3.0) and amounts to a 
percentile gain from the 35th to the Alst. 

For the first starting sites in cohorts 3, 4, and 5 there is a 
similar improvement from 54. 5 (N = 854) to 54.9 (N « 593) to 55.9 
(N = 482). The gain from 54.5 to 55.9 is statistically significant 
at the .002 level. 

ARITHMETIC 

K-Starting Sites 

The analysis of WRAT Arithmetic data by cohorts (Tables 4.5 and 
4.6) shows a very substantial improvement over cohorts in kindergarten. 

1C\5 
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Discounting the first cohort > which moved through a faster program and 
for .whlr.h the is small and restricted to four sites, the magnitude 
*^f the improvement is about one^-hdlf Standard deviation. Although the 
data for the last two cohorts is not yet in, it would appear that the 
same magnitude of improvement over cohorts will hold at the end of 
third grade. 

The analysis also shows a consistency across cohorts in the 
starting baseline performance and in the gains against the norms by 
grade level. On the WRAT Arithmetic tests, there is a sub6::antial 
gain during kindergarten (one-and-one-half standard deviation units) • 
A little of this Rain is lost in f irst grade ,-^<^-quite a. bit is lost 
in second grade. But there is a recovery of" small gains against the 
norm group in third grade, so that the net ^gain (K-3) against the norm 

group is approximately one standard deviation for later-starting 

/ 

cohorts. A loss in second grade (relative to the norm group) on WRAT 
Arithmetic has been noted in prior reports on E-B Follow Through kids. 
This loss is largely due to the WRAT emphasis on column functions at the 
second grade level, while the DISTAR program (first edition) emphasizes 
row functions. This difference is corrected by the end of third grade. 



Table 4.5 and 4.6 



Ist-Starting Sites / 

The arithmetic data from Ist-starting sites shows an improvement 
over cohorts of more than one-fourth standard deviation by the end of 
third grade. The superiority of cohort 1 at the end of kindergarten 
was not maintained for second and third grade. These findings are 
consistent with the program modifications that were made. 
O IP- 
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Table A*5 

Norm-Referenced Gains (Losses) on WRAT Arithmetic by Cohort 
from Kindergarten through First Grade for K-Scarting, 
Low-Income, E-B FoHow Through Students 



Kindergarten 



First Grade- 



.2 

a. 



CO 

c: 

5? 



CO 

o 
m 



c: 

€.3 





Cohort 


1 




PRE 


K 




POST 


K 


Post 


N 


99 


Post 


%.tjle 


63 


Post 


S. S. 


105 


Post 
ERiCit 


S.D. 
G. L 


16,8 
1.25 
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- , ' : 1 1 

! I ■ ! 
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5 


6 


1 


2 


3 


4 


5 


6 


K 


K 


K 


K 


K 














K 


K 


K 


K 


K 


K-1 


K-1 


K-1 


K-1 


K-1 


K-1 


757 


1185 


1172 


1051 


925 


279 


760 


1135 


992 


889 


754 


55 


66 


73 


11 


75 


55 


55 


61 


70 


70 


68 


102 


106 


109 


111 


110 


102 


102 


104 


108 


108 


107 


13.5 


U.3 


14.6 


15.4 


15.0 


10.8 


11.4 


10.9 


12.1 


11.4 


12.1 


1.16 


1.32 


1.A2 


1,51 f»L.43 


2.05 


2.11 


2.19 


2.40 


2.44 


2.34 



Table 4.6 



Pai?e 73 



HoTO-Rcferenced Gains (Losses) on WRAT Arithmetic by 
Cohort from Post-First Grade to Post-Third Grade for 
K-Starting/Low-Incoae, E-B Follow Through Students 



09 
CO 

C/7 



<2 

CD 



CO 



era 

' cs 

CD 



o 

02 



■Second Grade 



89- 
84- 
77- 
69 
60 
50 

40 

al- 
ia- 

16 

11- 

7- 
4 



Third Grade 




Cohort 


1 


2 


3 


4 


5 


1 


2 


3 


4 


POST 


1-2 


1-2 


1-2 


1-2 


1-2 


2-3 


2-3 


2-3 


2-3 


Post N 


233 


656 


796 


653 


750 


197 


408 


669 


715 


Post % til8 


42 


42 


50 


53 


55 


47 




55 


58 


Post S. S. 


97 


97 


100 


101 


102 


99 


100 


102 


103 


Post S.D. 


6.5 


8.5 


9.3 


9.8 


9.2 


8.6 


8.5 


8.5 


6. 7 


ER^ost G. E. 


2.7r 


2.82 


3.07, 


3.11 


if 


3.70 


3.84 


4.06 


4.13 
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I The gain against the nona group 1*3 Consistently shown to occur In 
first and third grades, not second. These gains are consistent with 
the findings for K-atarting sices. 

Tables A , 7 and 4.8 

MAT 

An improvetnent over col.orts In arlthroetic skills la shown on the 
MAT. For K-starting sites ai ;he end of third grade in cohorts 2, 3» 
and A, the Improvement on Total Math is from a mean standard score of 
66.3 to 72.6 to 72.7. In percentiles, this is an increase* from 35 to 
59 or one-half standard deviation unit- For Ist-starting sites at the 
end of third grade in cohorts 3, A, and 5, the improvement on Jotal Math 
is from a mean standard score of 68.7 to 70.7 to 71.2, In percentiles, 
this is an increase from Al to 32. This is a one-quarter standard 
deviation improvement. 

SPKLLIHC 

K-Start ing Sites 

In manv '-j-'vs, spelling was neglected by the sponsor in the process 
of giving priority to readia>;, oral communication skills, and lugical 
and mathematical competence. The locaL projects were encouraged to 
provide their own jpeliing programs. Some did, but many did not. After 
several years of failure to implement spell:. g programs, our project 
managers were en ouraged t ' be ^^re spelling was include^i f^t aid levels 
in all sites. The effects of tnl« effort are particularly apparent 
on the WRAT Spelling test which was given to all cohorts. Tables 
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Table A. 7 

\Som-Ref erenced Gains on V/RAT Arlchmctic by Cohort 

frocs Pre-First. Grade co Post-Second Grade for 
l3t-Start Ing, Lcv-incotne» E-B Follow Through Students 
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Table 4,8 

Nora-Referenced Gains on WRAT- Arithmetic by Cohort ^ 

from Post-Second qrade to Post-Tbird Grade for 
Ist-Startliig, Low-Incoiac, E-B Follow Through Students 
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4.9 gnd ^.10 show the data for each grade level. The lni[/roveinent over 
cohorts is ni^arAx^^one-standard deviation In kindergarten. The magnitude 
of Improvement then decr'^e^ses in successive grade levels, so that by the 
end of third grade, no significant effect is apparent (with or without 
projections for cohorts 5 and 6). Better ways of teaching advanced 
spelling skills are needed and are currently a programmatic concern 
of the sponsor. 

A consistency of gains against the norm group is present over 
cohorts. Substantial gains ;^re made *n kindergarten (piore whan one-and 
one-half staddaj:d deviation units for some cohorts) and then there is 
a loss of gapis, dropping back to the 56th p'ercentile. 

Tables 4?9 and A. 10 

t 

Ist-Starting Sites . . 

Tables 4.11 and 4.12 show the improvements over cohorts for Ist- 
starting sices. Small trends for improvement are shown » about one- 
fourth standard deviation in magnitude. 

Tables 4.11 and 4.12 > ^ 
\ 

MAT 

HAT Spelling scores from K-sites^at '"-he end of third grade show 
no systematic Improvements over cohorts 2, 3, and 4, For Ist-sites 
an improvement of less thAn one-fourth standard deviation is projected. 
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Table A. 9 



Norm-Referenced Gains (Losses) on WRAT Spelling by 

Cohort^ from Kindergarten througWFirst Grade for 
K-Starting, Low-Income, E-B FoUow Through Students 
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Table 4.10 

Norm-Referenced Gains (Losses) oii WRAT Spelling by 

Cohort from Post-First to PostAxhird Grade for 
K-Starting, L9w-Income, E-B Folldw Through Students 
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Table 4.11* 

Norm-Referenced Gains on WRAT Spelling by Cohort from , 
Pre-First Grade to Post-Second Grade for lst~Starting, 
Low-Income, E-B Follow Through Students 
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Norm-Referenced Gairis on VTRAT Spelling by Cohort 

from Post-Second Grade to Post-Third Grade for 
lst~Starting, Low-Income, E-B Follow Through Students 
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IQ GAINS 

An examination of IQ gains using the SIT over cohorts will not 
be undertaken for K-starting sites because of the limited data in the 
PRE-POSf GAINS analysis* For the Ist-starting sites, the gains data 

are shown by cohort in Table 4,13* The cohort 1 data is very limited' 

i 

(based on only one site where the entry level of the children is quite 
low). However, this cohort shows a level of gains that was not shown 
• again until cohor^ 5. From cohort 2 to cohort 5 some improvement in 

gain is to be not^d. The difference in gain by cohort 3 over cohort 2 and 
by cohort 5 over cohort 4 are significant beyond the .001 levtil. The size 
t>f the gains appear to be correlated with entry level IQ. 



Table 4,13 



\ 

t 

\ 



\ 

\ 



Summary 

The analyses by cohorts show that the cohort 1 (though based on 
limited data) was somewhat superior in progress to the next two. The 
program changes introduce to insure that a^l low performers were taught ifiay 
have actually slowed down some better students^ and led to lower mean 
outcome levels. Nevertheless, cohorts 2 to 6 show a progressive 
improvement in performance in K-sites on WRAT Reading and Arithmetic 
(and to a lesser degree Spelling). Some progressive improvements on the 
MAT are also shown for the cohorts teste^ with the MAT. The analysis 
-revealed that the major source of the improvement over cohorts was 
associated with improvements in implementation at the kindergarten level. 
It took some time to get kindergarten teachers to make more serious efforts 
to teach academics to young children. Many had been trained that it was 
* quite inappropriate to do so. 
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Table 4.13 

SIT IQ Gains over Cohorts for Ist-Startlng, - 
Low-Income, E~B Follow Through Students from 
the PRE-POST GAINS Analysis 
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In keeping with this, less pronounced improvements over cohorts 
2 to 6 were found for Ist-sites. The largest improvement was found in 
arithmetic, where successive program modifications may account for 
the gains. / 

The PRE-POST GAINS analysis was used to examine IQ gains for Ist- 
starting sites only. Discounting the cohort 1 which showed a nice gain 
(7.35) for 57 children in one site, the improvement gains from cohort 2 
to 5 was from 2.76 to 8.73. 
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CHAPTER 5 

CONTROLLED WITHIN-SITE COMPARISONS (INCLUDING HEAD START) 

In a number of our projects we started at several grade levels at 
once, or th6 sites switched from being Ist-starting to K-stafting after 
Follow Through was under way. Where these /^vents happened, it is 
possible to make comparisons within the same^ schools with the same- 
teachers for children who have had two years of the E-B program, three ^ 
years of the program, or four years of the program. If the program is 
having an impact, those children who are in it longer should do better. 
These comparisons also provide another kind of answer to the question 
"Does kindergarten make a difference?" Since the comparisons of K- 
starting sites with Ist-starting sites are pessibly confounded by regional 
differences in socio-economic conditions, these wi thin-site comparisons 
permit an assessment of the possible gains to be made with the dis- 
advantaged by a fuller use of kindergarten for basic instruction. 

In this chapter we will also examine data for three sites where 
Planned Variation Head Start was introduced in 1969, 1970, and/o?r 1971. 
Examination of the progress of kids with and without a sponsored 
Head Start program is possible. In addition, it is possible to evaluate the 
effects of sponsored Headstart programs for five-year-olds versus 
sponsored public school kindergartj^n for five-year-olds. 
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CHEROKEE 

The first wlthln-slte comparisons coma from the Eastern Band Che okee 
Indian conmunity in North Carolina where the children attend a BIA school, 
n 1970, we installed our program simultaneously in kindergarten, first, 
and second grades. The children had had the benefit of Follow Through 
theXyear before with another sponsor who was terminated. As far as we 
could\ determine, thsre had been no significant impact of that year of . 
Follow\Through on the school program. Tables 5.1 to 5.5 provide the 
comparisons on WRAT and MAT variables for the children who started the 
program i\^ second grade (2 years of E-B) , first grade (3 years of E-B) 
aAd kinder^rten (4 years of E-B) . The Restarting group includes four 
cohorts through first grade, three through second, and two through third • 

Examination o£ the K-starting data by cohorts shows progressive improvements 

\ ' # 

for the first tl^ree cohorts. 

\ 

Table 5.1 gWes the data for WRAT Reading. It can be seen that 
the children with V years of program are nearly a standard deviation above 
the children with 2\years of program. The difference is highly 
significant (p < .Oo\). Children with 4 years of program are also well 
ahead of those with on^y 3 years of program at the end of first and second 

grades, but not third g^rade. Apparently 3 years was enough to catch up 

-■ \ 

on decoding skills. . 



Table 5.1 
Table 5.2 
Table 5.3 
Table 5. A 
Table 5.5 
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Table 5.1 

Comparison of Progress for All Cherokee Students with 2, 3, and A 
^ Years of Follow Through on WRAT Reading 
(Retentions with Agemates) 




Table 5,2 



Comparison ot Progress for All Cherokee Students wuh 2, \. and < 
Years of K-B Follow Through on WRAT Arithmetic 
(Retentions with Agem.ite^) 
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Table 5.4 . . * ^ 

Coaparlson of/#focress for All Cherokee -Students with 3 and 4 
Yo^ra of%E-B FtSllow Through on HAT. 

Total Reading and. Total M0th 
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.Table 5-5 



Coffipariison of Progress for All Cherokee Scudencs with' 3 and A 
Yearf. of E-B Follow Through ori-MAT 



Language 



and 



SjJi^lling 



.IS- 

'an* 

« 

m 
«^ 

V5. 



i/7 



i9r 

60 

SO 
40 
31 

23 

11 
7- 



4 



1 



3 S/ears 



4 vjears 



5b 

3 vjears. 




post 


3 


3 


H 


106 


167 


%-tilB 


.. AO 


67 


s. s.. 


65.3- 


75.7 


s.o. 


10.7 


12.2 




■3.69 


5.03 



* MRAT Cour-year baselines used* 



J 


1 


1 


103 


246 


184 


16 




43 


59 0 


57.6 


62.3 


11.2 


10.7 


12.3 


1.27 


3,11 




■12-i 







. As Table 5.2 showk,, the A-year students did significantly better 
(£ 4L .001) than both the 3-year group and the 2-ycar groups on^WRAT^ 

Ariths&eticf. Table 5.i shows that this same conclusion holds for WHAT 

•> . . 

Spelling* The aagnitude of the differences at the^ end of third grade 
are from a quart* to a half standard deviation. 

The coaparisona for MAT Total Reading and Total Math (Table 5.4) 
also show significant effects favoring the 4**year group. The difference 
"^f or MAT Total Reading is significant at the .002 level, and that for 
Total Math is beyond the .001 level. Table 5-5 shows the effects for 
Language and Spelling on the MAT. The Language difference sijgnificantly 
favors the A-ycar group < .001) as does the Spelling difference 
(£<V02). ' ^ 

IQ data arc available for comparing the 3-year group with the 

A-year group.. The cotsparisons are based only on the students for whqjn 

there is a pretest and posttest. The data are as follows: ^ 

.Pre-1 Post 3 - Gain 

Mean^ SD -N Mean SD N Mean SD N 
3Ayear group.> 98,6 17.3 75 101.7 18*6 75 3.05 12. A 75 

Pre-K Post 3 Gain 

A-year group L 101.4 10. 0 A9 . 107.9 15.5 A9 9.38 13.5 73 . 
A-yeaf group 2. 100.5 13.7 24 115.7 13.9 24 / 

The difference in IQ gain is significant at the .01 level. The 
difference at the end of< third grade is also significant. 

These comparisons clearly show a strong program impact that increased 
the longer the program was in effect. 



P?ge 93 V 

1^ 



. E> ST. LOUIS 

. — ' 

St. Louis, Illinois is a Black community just over the river 
from St. Louis. E. St. Louis started Follow Through in 1968 as a' ist- 
graders tar ting parent-Implemented program who chose the' E-B model as 
sponsor. Jn 1970, public school kindergartens^^were introduced. This 
gave us three cohorts with Ist-startlng children and four with K- 
starting children. Jn .addition. Planned Variation Head Start wa,s 
Itttroduced as a program for five-year olds in 1969 and for four-year 
old§ in 1970. The Head Start data will be discussed later in this chapter 
For now, the children with E-B Sponsored' Head Start are excluded 
from the analysis^ 

V It was noted in discussing the data for Ist-starting sites over 
cohorts, that cohort 1 for E, St. Louis performed unusually high on 
most measures at the end of third grade. They had been taught' by a 
selected group of teachers and made great progress; This unusual 
performance for cohort 1 tends to obscure some of the effects of K- 

.starting and Ist-starting cohorts. Tables 5.6 to 5.10 present the 
comparisons for children with 3 and 4-years of program. For WRAT 
Reading, the differences are significant and' favor<K-s tar ting .children 
at the end of grade i and 2, but not 3. The\, unusually, high performance 
of cohort 1 children at the end of grade 3 reduced the difference. 
Separated by cohort the means at the end of third grade arer 
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Table 5.6 

\, . 

Compari^Son of Progress for tow-Income E. St. Louis Students 
with* 3 arid 4 Years of Et»B^To11ow Through on WRAT Reading 
» » (Retentions with Gradekates) 
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*Four-year pre-K baseline. 
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Cohorts S And 6. wlll/be stronger than the Restarting cohorts 3 and 4 il- 
curreht trends continue.,. Thus, it iis apparent that the performance of 
cohort 1 on WRAT Reading is atypical. If thefee data :were to be excluded 
•from thQ evaluation, there would be an advantage for the K-starting ^ 
'cohorts of three-fourths of a standard deviation (112 versus 123). 

On WRAT Arithmetic, cohort 1 is nearly a grade level ahead of cohorts 
2 and 3 (two-thirds standard deviation) but even given this atypical 

performance, the K-starting cohorts show a significantly superior perfbrmance 

^ . : " : ' ^ ' ' '^"^ ' 

at all grade levels (£ <.001). Excluding cohort 1, the superiority is 

about .7 grade levels. ^ * 



Tables 5.7, 5.8, 5.9, 5.10 



On WRAT Spelling (Table 5.8), again th^ highly atypical performance 
of cohort 1 (cohort 1 - 112, cohort 2 « 99,,. cohort 3 - 94) eliminates 
any significant, effect by the end pf third grade. Comparisons excluding 
.cohort 1-are highly significant, as are effects at the end of first and 
second grades. * # ^ 

On the. MAT. the comparisons involved Ist-starting cohort 3 and with 
K-starting tohorts 3 and 4., The differences favor the K-starting groups- 
at the .001 level or better for Total Reading anJ Total Math (Table 5.9). 
The magnitude of the differences is about one-half s^tandard deviation or .4 
grade levels in each case. On MAT Language, ' there is^ no significant 
d^f ferenca, while the difference on Spelling favors the K-starting groups ^ 
at better than the .01 lev(al. ^, 

Comparisons on the SIT show more between-year gains on for K-starting 
than for Ist-starting children, but because the testing with the SIT does 
not cover enough of the same kids at pretest and posttest, they will not 
be reported. . * ■ . 
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Table 5.7 



Comparison of Progress for Low-Income E, St. Louis Students 
with 3 and 4 Years' of E-B Follow through on VfRAT Arithmetic 
(Retentions with Grademates) 
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Table 5.8 



Comparison of Progress for Low-Income E. St. Louis Stiid, it's 
^th 3 and 4 Years of E-B Follow Through on WRAT Spelling 
(Retentions with Grademates) 
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Table 5.9 > ' 

Comparison of Progress for Low- Income E. ,Sc. Louis Stude'ncs 
with 3 and A Years of E-B FoHow Through on. MAT 
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Table 5^10 



Comparison of Progress for Low-Income E. St." Louis StudentSv 
with 3 and 4 Years of E-B Follow Thrpugh on MAT 
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•^"^ In .suicmarjr, the* E. St. Louis data siipport a strong effect of 

^ ' " \ ^ ' * 

starting the program in Kindergarten* if the usuallyhigh performance 

of Jibhort 1 is discounted. \ ' . 

rilPPIN 

^Flippin is a small Tural White community In northern Arkansas. In 
1969 we initiated tl^'e prpgram in Fllppin for kindergarten, first, and 
econd grades. This gave us another chancei to compare tne effects of 
2,\ 3, and 4 years of pr6gram. As can be no\*:ed from the average beginning 
K baselines in'Table 5.11 and 5.12, the Flippin children en\er with more 
skirls, than found for our typical disadvantaged children. They also 
progvkss much further. ' 

Tables 5.11 and 5.1^ show the comparisons on. WRAT Reading ,\ Arith- 
metic, ^nd Spelling. Note that we have changed the data being displaye d 
from tha\ ;3hown for 'Cherokee and E.. St. Louis. w F6r the first two\ within- 
site scudfes, we wanted to get across the point that the differences 
attributable to an early start in the program produced larger effects 
the. earlier igrades than remained by the end of third grade. This satae con- 



clusion holdslfor all -the sites being compared. The remainder of the\ 
comparisons (except for Headstart) will ^cus on end of third 'grade 
comparisons onlV and will display the data for each cohort. Tliis allow\3 
the reader to see consistencies or trends acros's cohorts. 

For each of the Flippin comparisons, the average 4-year program 
effects are signijr\Lcantly above the average 2- and/p-year effects. For \ 
WRAT Reading and Ar^^thmetic, the significance is beyond the .001 level. ^ 
For .Spelling it is b^^yond the .01 level. The Reading scores show pro- 
gressive improvementa which are educationally significant. Arithmetic shows 
a significant gain in ^ going from 2 to 3 years of the program, ,but not to 4. 



\ 
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• \ » Pase 101 

The resiilts/ for Spelling are like those for Arithaetic. Ihe Flippin 
children are performing well above grade level in all areas. ^ 



Tables 5.11 and 5.12 



RACINE 



Data comparing 3-and A-year effects from Racine, Wisconsin, are 
presented in Tables 5.13 and 5.14. The difference for 3-and A-year 
effects are each significant. For Reading, the significance is beyond 
the .05 level, for Arithmetic and Spelling the significance is beyond 
the ..dOl level. After the co*hort 1 surge which we have seen many times 
b&fore, then is progressive improvement over cohorts 2 to A. The' 
magnitude of the difference in effect for K-starting and Ist-starclng 
cohorts is o.. the order of one-half to three-fourths standard deviation. 



or .6* to 1.0 grade equivalents. 



Tables 5.13' and 5.1A 



PROVIDENCE 



Data comparing 3-and 4-year effects in Provi^nce, Rhode Island 
are shown in Tables 5,^15»and 5.16. The differences in overall effects 
are each significant beyond the .001 level and amount to at least one- 
half standard deviation. The children with 4 years of the program are 
.6 to .8 grade levels ahead of those with only 3 years of the program. 





Tables 5.15 and 5.16 
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Tabic 5*11 

C^cparlson of Frogretis at the Kn4 of Third Crude for 
Low-Incocc Studentar froa Fllppln nith 2, 3«^' and 4 Years of S-B 
Follow through on WHAT Reading and Arithsctlc 
(Retentions^ with Ageaateo) 
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Table 5.12 



Cojaparison of Progress -at the End o£ Third Gr-a^e.for 
Lou-lncone Students froci Plippln with 2, 3» and ^^chrs of E-B 
Follow Through on WRAT Spelling 
(Rctenttohs «lth Agcnaces) • 
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Coaparisorv of Progress at the End of Thttd C^de for 
Lov'-'lnco?3e Students from Racine* wltK 3 and 4 Yearns of E-B 
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Table 5 -14 

, Comparison of Progress at the End of Third Grade for 
Low-Income Students froro Racine with' 3 ^nd A Years of 
FbUow Through on WRAT Spelling 
(Reten^tiorts with Agemates) 
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Table 5.15 

Comparison rff Progresls at the End of Third Grade for 
Low-Income Students from Providence with. 3 arid 4 "Years of E-B 
Follow Through on WRAT Reading and Arithmetic 

(Retentions with" Agemates) ' • ' " 
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Table. 5. 



Low-Income 



Comparison of Pw)gtess at the End pf Third^Gr^de for | . 
ncome Students from Providence with 3. and 4 ^^ears of ^E-rB 
EoMow Through oh WRAT Spelling 
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ROSEBUD 



The Rosebud Sioux Indians are represented in our model by four 
separate school systems. Three of the jurisdictions are funded through 

■a . 4 * ^ 

the Rosebud Tribe and are grouped for this presentation. They are the 
St. Francis Mission School, the Millette County schools at White River 
and Norris, and the Millette County school at Wood. The fourth juris- 
diction is the Todd County Independent School District (presented next). 

Cohort 1 includes Ist-starting children at St. Francis. Cohort ,2 
includes both Ist-starting and K-starting children at St. Francis and in . 
the Millette County schools. Each of the comparisons of the average 
3-year and A-^year effects is significant beyond the .001 leve^r^T^rogx^sive 
improvements over K-starting 'cohorts 2, 3, and 4 are very impressive for 
both Reading and Arithmetic. The magnitude of the effec^ts.are on the 
order ofv.,a full grade level improvement. 



r 

Tables 5.17 and 5.18 
I \ TODD COUNTY 



The final comparisljns^re for the mostly Sioux children in five 
Todd County public schools. The overall comparison of 3-year and 4-year 
effects is significant only ^'or Arithmetic (£ < .001). Reading shows a 
progressive improvement over cohorts 2 to 4, and both 3-year and 4-year 
cohorts perform^t an impressive level. The^^prjgressive improvements in 
Arithmetic represent a gain of more than one-half standard deviation. 
Spelling shows no improvement over cohorts. ^ 



Tables 5.19 and 5.20 
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Comparison of Progress at the End .of Third Grade for 
Low-Income Students from Rosebud with 3 and A/O^ears of E-B 
Follow Through on WRAT Reading and •Arithmetic 

(Retentions with ^g^mates) ' • . , 




ERLC 



Cohort 


- 1 


2 


2 


3 


it. 


1 


2 


2 






POST 






















3 


3 . 


3 


3 


3 


3- 


3 


3 


3 


3 


N 


23 


60 


32 


52 


54 


23 


60 


32 


52 


54 


%.-tile 


73 


50 


70 


77 


91 


23 

/ 


32 


37 


50 


61 


S. S. 


109 


100 


108 


111 


120 


89 


93 


95 


100 


, 104 


S.O. 


-19.3 


11.9 


14.0 


17.3 


<21.5 


12.2 


8.5 


8.0 


7.8 


f.9 


G.E. 


4.69 


4.01 


4.59 


5.02 


5.76 ird 


2.76 


3.30 


3.36 


3.97 


■ 4.14 



V 'r- Page^llO 



fable 5.18 



CO / 

o 
CO 



l£2 
-o 

CO 
CO 



o 



CO 



03 

o 

a> 
ex. 



. Comparison of Progress at the 'End of Third Grade for 
Low-Income Students from Rosebud with 3 ,and A Years of E-B 
Follow Through on WRAT Spelling 
(Retentions with Agemates) ^ 
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Table 5>J.9 



Comparison of Progress at the End of Third Grade, for ^ 
Lowylncome Students from Todd County with 3 ^and 4 Years of E-B 
Follow Through on WRAT Reading and Arithmetic 
* * (Retentions with Agemates) 
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Table 5.20 



Comparison of Progress of the End of Third Grade for 
Low-Income Students from Todd County with 3 and 4 Years of iE-B 
Follow Through on Spelling 
(Retentions^with Agemates) _ 
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Summary on Vt'hln-Slte Comparisons 
^These withln-slte comparlsohs based on different durations of program 
implementation demonstrate an advantage to 4 years of program over "3 

years of program on the 6rder of one\half staadar'd deviatioji , or .6 to 

• \ \ • ■ 

.8 grade levels. ^ Larger differences afe present in some cases ano 

insignificaivt differenceiJ occur for a feWvcomparisons, The overall 

findings are consistent with the previous ciimparison made between K- 

atar.ting and Ist-starting sites, and support hhe concluilon that the 

differences reported earlier cannot be attribut\^ to regional difference? 

«" , " ■ 

in sl;udents. , i ; 

PLANNED VARTATTON HEAD START COMPAI^ISONS 
We now wish to examine how the above findings within Follow Through 
compare to those combining Planned VariationHead Start with Follow through 
Planned Variation Head Startwas operated through Community Action Agencies 
rather than the public schools, although th% .programs were often housed in 
public schools. 

In the years 1969, 1970, anr. 1971, sponsored Head Start programs ^ 
were initiated in three E-B Follow Through Communities. The coianunicies 
were Tupelo, Mississippi; E. Las Vegas, New Mexico; and E. St. Louis, 
Illinois. All programs bagan With five-year olds. In tha second year of 
the Planned VariationHead Start in E. St. Louis, the program was moved to 
four-year olds because public school kindergartens were initiated. The 
■ program in Tupelo covered 3 entry groups, while- those in E. Las Vegas and 
E. St. Louis~^overed only 2. The comparisons ve will present will look 
primarily at performance at the end of third grade. However, school 
entry performances, with and without Head Start can also be examined, as^ 
well as the progress by grade level. 
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Tupelo^ 

Tables 5.21 to 5.25 , present the comparisons for Tupelo. On 
entry to. first grade, there is a significant superiority of the Head Start 
students over the rion-riead Start students. At the end of third grade, 
the children who had Head Start are ahead on all eight measures, and six 
of the differences are significant. The levels of significance are an 
follows: < 
WHAT , ' ' 

Reading .01 " 

, ^ Arithmetic .001 ' , , 

Spelling .001 



MAT 



Total Reading .001 . ' ^ . 

Total Math .001 

Language N. S. 

Spelling . .001 
SIT IQ N. S. ' ^ 

The magnitude of the significant differences are from a quarter to / 
a half standard deviation, or from .4 to .6 grade levels. 



Tables 5.21 to 5.25 




\ 



\ 



i V 
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^■\ Table 5.21. 

^ v^^Tiipelo Head Start and FoiJow Through Comparisons on 

WRAT I^eading for Low-Income Children, Retentions" with Gradeoates 



CO - 

oo 

c= 
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60 
50 
40 
31 
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11 
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x3 
t 
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HS 
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2 


3 




1 


*2 




65 
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8A 


110 


139 


364 


296 
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5 


kl 


66 


66 


^66 


10 


37 


50 


53 


76 


97 


106 


106 


106 


81 


95 


100 


101 


13.2 


17.9 


20.2 


18.5 


18.2 


13.0 


16.3 


17.8 


17^3 


-.16 


.83 


2.20 


3.40 


4.42 


.35- 


1.59 


3.04 


3.91 
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'X 5 



i 
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,Table 5.22 

T.upelo Head Start and Follow Through Comparisons on 
WRAT A?:lthmetic for Low-flncoxae Children, Retentions with Crademates 
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CO 
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o 
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16- 



n- 

7- 
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N 
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, HS 
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■ 2 


3 
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84 


110 
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3b J 


296 


21^ 




7 


53 


' 55 


J7 


50 


21 


jJ 


30 


39 , 




78 
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95 
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88 


95 




96 




15.4 


14. 3 


10.9 


7.0 


8.2 


11.2 


U.9 


8.3 


S.6 




-.lA 


liOA 


1.90 


2l;o 


3.90 


.65 

> 


1.54 


2. 38 


3.49 
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ML' 



. * Pane U7 ^ 

4 — \ . 
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Tupelo Head^ Start and F.ollow Through Coaporiaons .on 
WRAT Spelling for iov-Inco»e Children^ detentions with Grai^ctaatcs 
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Tabl<^5i24 

Tupelo Head Start and Follow 'Hirough Cooparl^ons on 
"mat at End of Third Grade for Low-Inco«i5 Children 
(Retentions «tth -Gradccjates) 
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Math 
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anc 
FT 
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HS 




HS 


I 






HS 


I 
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• 3 


3 


3 


3 


3 


3 


3 


3 


H 


109 
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80 


110 


81 


10? 


74 


%-tite 


32 


17 


i.0 


28 


52 


50 


lit* 


28 




^55.4 


50.6 


68.0 


64.1 


70.8 


69.8 




56.5 


S. 0. 


8.0 


7.3 


10.0 


9. a 


* 12.2 


10.7 


11.7 


10.5 


8. E. 


3".d4 


2.59 




3.31 


A. 48 


4.22 


3.64 


3.02 
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Table 5.25 



PRE 
' POST 
N 

%-tj|8 

s. s. 
s.o. 



Tupelo Head Scare and Follow Through Comparlsoiis on 
SIT IQ's for Low-Income Children, Retentions witlKarademates 
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3 




1 


2 
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231 


3a 


29 3 
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34 




58 


47 


38 


26 


35 


40 


35 


93.8 


101.6 


n.l 


^8.8 


95.7 


90 - 


94,4 


96.5 


94.2 




15.3 


14.8 


14.3 


16.0 


13.0 


13.8 


13.8 


16.4 
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E« Las Vegas 

fables 5,26 to 5.30 present nhe Head Start comparisons for E. Las 
VeRas. Again, on entry to first grade there are clearly significant 
differences favoring the Head Start children. These differences are 
reduced in size by the end of third grade, but 5?till all eight 
measures* favor the Head Start children and 5- of the 8 do so significantly. 
The levels of significance are as follovfs: 

WRAT 

Reading N. S. 

Arithmetic .05 ' * ' 

Spelling N. S. 

MAT . ' * . ^ 

Total Reading .05 

Total Majh .01 

Language • .01 

Spelling N. S. 

SIT IQ .001 

The magnitude of the significant differences are on the order 

\ 

of one-quartec standard deviation, \>r .2 to- .4 grade levels. 



Tables 5.26 to~^.30 



1 



\ 
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Table' 5.26 



' E. Las Vegas Head Start and Follow Through Comparisons on - 
. WRAT Reading tor Low- Income Students, Retentions with Grademates 
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S. D. 
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12.5 
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16.4 


G.E. 


.11 


1.37 


2,64 


3,94 


5.06 


I5i 


.53 


2.46 


3.93 


4.93 
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Table 5^27 



E. Las Vegas Head Start and Follow Through Comparisons* on 
WRAT Arithmetic for Low-Income Students, Retentions with iSrademates' 
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61 
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68 
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95 
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99 
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S. D. 
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7.9 
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12.2 


10.7 
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8.4 




-.OA 


1.21 


2.29 


3.04 


4.19 


.89 


-■2.24 


2.88 


4.03 
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£• Las Vegas Head Start and Follow Through Comparisons on 
WRAT Spelling for Low-Income Students, Retentions with Graderaates 
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1.95 


2.88 
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o 
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84* 

77- 
69 
60 
5p 
40 
31- 
23- 
16 
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Table 5-29 

E. vLas Vega^^Head Start and Follow Through Comparisons on 
jVT at Epd of. Third Grade for i'ow-Ihcome Children 
(Retentions with Grademates^) . 
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Reading 
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Math 
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-aft 
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97 
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G. E. 
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3.38 


A.A5 


4.12 


^ 5.38 


4.76 


3.61 


3.78 



15 



PjaRe 125 



O 
CO 



CT3 



o 



-a 
en 

•4-* 



o 

c: 



to 



o 

a> 
cu 



PRE 
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N 
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' Table 5.30 

E'. Las Vegas Head Start and Follow Through Comparisons on 
SIT iQ's for Low-Income Children, Retentions with Grademates 
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3 
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S. S. 100.8 106.3 ]0*>.i 104.5 111.8 
S. D. 14.6 13.6 13.2 12.0 14. A 



- 1 2 3 
267 384 262 249 
50 53 68 63, 
100.2 101.9 107.2 105.2 
11. .9 13.0 13.0 15.6 
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E> St. Louis 

Tables 5#31 to 5.35 present the comparisons for St. Loylfs five- 
year-old Head Start. The five-year-old Head Start fs compared with the 
lst-8tartlng Follow Through children. Comparisons at entry to first 
grade are not. available. Seven of the eight differences at the end of 
third grade favor the Head Start children and three are .significant. ^The 
levels of significance are as follows: ^ ^ 

WHAT 

Reading N. S. 

Arithmetic .003 
/ Spelling N. S. (favors Is t-s tar ting Follow Through) 

^ MAT • ^ 

Total Reading -.01 
Total Math K."s. 

Language N. S. * 

Spelling .01 
SIT IQ ' N. S. 

The magnitude of the significtant differences are on the order of a 
quarter standard deviation, or from .2, to .4^ grade levels. 



Tables 5.31 Co 5.35 



Tables 5.36 to 5.40 present the jcomparisons for th^ four-year-old 
Head Start children with the K-starting Follow Through students. While 

US' 

there Is a superiority of the Head Start children on entry to kindergarten 

there are no significants differences at the end of third grade. We would 

> J- 

caution the reader not to conclude that a program for four-^year olds is 
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Ta6le 5.31 
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E. Sc. Louis Five-Year Old Head Start and Follow Through Comparison 

' WRAT Reading 
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*E. St. Louis pre-K Follow Through baseline. 
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E St. Louis Five-Year Old Head Start atid Fellow Through 'Comparison 



,WRAT Arithmetic; 
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*E. St. Louis pre-K Follpw Through baseline. 
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Table 5.33 

E. St. Louis Five-Year Old Read Start and Follow Through Comparison 

WRAT Spelling 
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*E. St. Louis pre-K Follow Through baseline. 
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Table S*3A 



Sc. Louis Flve-Vcar Old Head Start and Follow Through Conparison 

MAT 
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E. St. Louis Five-Year 014 Head Start and Fo'llou Throueh.CoBparison 
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not of value. This prograa was not «eil~lopieaetited, aide tu^overs and 
abseosca were extreoely high, and actual j-rogresH in the progrUs. very 
low. The proper conclusion Is that the possible value of a fodr-year-old 
prograa has not been dcisonatrated. 



Tables 5.36 to 5.40 



l^hen the f ivc-yeAr-oid Bead Start prograa Is cospared with the public 
Etchooi kindergarten prograia, the kindergarten children arc clearly 
ahead at the end of the first year as follows: ' , 

Standard .Scores 



WRAX 

* . ' Kcadlng 
.Arithaetic 
Spelling 
SIT IQ ■ . 



Post as-5 



.93 
103 
88 • 
105.8 



Post FI-K 



112 
112 
109- 
107.9 



001 
,001 
.001 
K. S. 



At the end of third grade differences are as follows: 



«RAT 



Reading 

Arithssetic 

Spelling 



HS-5 



125 
100 



MAT 



Total Reading 56.5 

Total Katb 70.4 

Language 72.9 

Spelling 64.7 



SIT IQ 



105. 1 



PT-K 
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106 



59,^ 
72.6 
71.8 
63.2 

m.a 



Direction 



US 
FT 



FT 
FT 

as 

HS 
FT 



N. S. 
N. S. 
.03 



H. S. 
H. S. 
N. S. 
K. S." 

.05 



Five dlffercocea favor the five-year oida taught in Follow Through 
(two aigalficantiy) and three favor the five-year olda taught in Head Start 
(none significantly). These data give a slight edge to the children taught 
in the , public school setting. ii-O 
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Table 5.36 ^ 

£. St, Louis four-Year Old Head Stare and Fallow Through Coaparlson 

WRAT Reading 
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St. Lowlf pre-K Follow Throo^Visf^eUne. 
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Table 5.37 

t 

E. St. Louis Four-Year Old Head Start and Follow Through Comparison 

WRAT Arithmetic • 
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Table 5.39 



E.^ St. Louis Four-Year Old Head Start and Follow Through Cpmparison 

MAT 



CO 

o 



■ 

Am. 

cz 

So 



CD 
CO 

o 



" Total 
Reading 



34- 
77- 
69 
60 
50 
40 
31- 



g 23 



16^ 



and 

f T. F. 



Total 
Math 



Language 



Spelling 



H. 

an0 



S.-4 



I. 



H. 

anc 
F. 



£ .-4 



H. 

anc. 



£ .-4 



11- 

7 



T 



ERIC 



PRE 


V 




K 


HS-4 


K 


HS-4 


K 


POST 3 


i 


■J 


3 


3 


3 


3 


3 


H 60 


Ihf) 


SA 


159 


56 


160 


56 


157 


%-U!e 36 




61 


57 


55 


55 


50 


48 


S. S.i7.0 




/ J, 


72. h 


71.7 


71.8 


64.2 


63.2 


S.O. 7.8 








10. A 


11. ' 


12.2 


11.5 


6. £, uVj 




6.20 


',.0b 


A. 42 


4.50 


3.84 


3.73 
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o 

CO 

.2 

CO 



CO 

•a 



CD 



to 



CD 



89- 
84- 
77- 
69- 
60- 
50- 

'40- 
31- 
23- 
16- 
11- 
7- 
4 



. Table 5.40 . ' 

E. Sc. Louis Four-Year Old Head StartTand Follow Through Corapari|son 

SIT ■ , 



i. S. jin 



d F. T. 



S3 '■' ^3 



F. T 



. only 



is 



I ERIC 



PRE 


HS 


I 


I i 


1 


K 










POST 


HS 


K 


1 2 


3 




K 


1 


2 


3 


H 


86 




75 


60 


^83 


587 


'■'05 


268 


159 


%-tlle 


68 




71 


78 


A6 


70 


65 


65 


79 


S. S. 


- 106.8 






1 - } 




1(1 V 


10:>.6 


10' ri 


M 1 .8 


S. D. 


I j.9 




-/ 15.9 


16.6 


12,7 


13.6 


14.3 


16.5 


18.5 



t I 
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Summary on .Head Start Comparisons 
Significant post- third-grade effects for Planned Variation Head 
Start children are present for all three sites with five-year olds, ka 
added year of teaching does lee.d to more proficient kids. Our inter- 
pretation of the four-year old H^ad Start results is that no conclusion 
is possible because the program was not well- implemented. 

A comparison of the magnitude of effects found for the vl thin - 
site comparisons of 3- and 4-years of program with the effects within 
site with and without a K-level Head Start is instructive. Th<^ average 
advantage by the end of third grade of the E-B Planned Variation Head 
Start programs was about one-quarter standard deviation. The average 
advantage for a 4-year Follow Through Program was about one-half 
standard devi'^^^lon. This finding is in keeping with, our experience as 
sponsors that it is much easier to impfement and maintain a program 
within a public^ school administrative system than within the framework 
of a Community Action Agency. These findings would support a recom- 
mendation to fund Head Start as part of public school programs . 

In a later chapter we will present an, analysis of thb-ways in which 

4 

student attendance, lessons taught, and other process variables relate 
to outcomes, it is within these analyses that support for the inter- 
pretation of the above conclusions about program implementation is found* 



CHAPTER 6 

FOLLOW UP STUDY. OF FIFTH AND SIXTH GRADERS 

^ In the sVrlng of 1973, fifth and sixth graders were tested with 
the WRAT (Levels 1 and 2) and the MAT (Intermediate) in seven sites 
where control groups could be found. In four sites the SIT was also 
given .to be used as a dependent variable. Only the fifth graders in 
Flippin and Flint had started the program in kindergarten. All other 
groups started in first grade. Because of this fact, the results to 
be reported renresent a low estimate of what might have been achicrved. 
The only selective factors operating in choosing «ites were the avail- 
ability of a control group and district cooperation. 

There were 711 Follow Through students and 766 Non-Follow Through 
students in the various comparisons. Low-Income students were sought 
in each case except Flippin, where all students were used because of the 
small N available and the probable equality of the two communities being 
compared (Flippin and Cotter). As it turned out, there were a few Non- 
Low-Income students in most groups, so Low-Income, Non-Low-Income was 
used as acovarfate. Other sample differences in education of parents, 
sexof student , number of siblings, and Ethnic group (White, Non-White) 
were also adiuated by the use of covarlance analysis where possible. 

In Tupelo, Mlaslaolppl, the local district California Achievement 
Teat results were used in place of the MAT and thp reading ocoreo were 



converted to KAT equivalents using information from the Anchor Test 
Study. In Flint, the test results for MAT Heading' were provided to us by 
the loccl district for "Follow Through and a comparison group. 

CovariancG analyses were computed for each variable for each site. 
This provided for a Total of 1A9 comparisons. Since the group si^es 
tended to be small (averaging about 50, but ranging from 18 to 117)" we 
accepted a one-tail .05 significance level. We also examined trends 
with probability levels of .15 (one-tail) or less. Uaing theu .05 
level (one-tail test), differences of approximately a quarter standard ^ 
dev;Vation would be significant when the two samplf. ylzes were 50* 

The ^students tested by site are listed in Table 6.1. 

The overall pattern of findings is shown in Tabli 6.2. 

Thft results show 53 significant differences out of a possible 1A9 

at the .05 level using a one-tailed test. Of these 53, 50 favor Follow 

Through and 3 favor Non- Follow Through groups. In addition the trends 

between .06 and .15 probability levels show 19 out ot 24 trends in 

♦ * 

fz|vor of Follow Through. These findings imply that the E-B Follow 
Through model has produced effects which are still detet:table two and 
three years after the special program is no longer in effect, and under 
conditions when there was no assurance of a building on the gains made 
m Follow Through. Furthermore, the effects come primarily from 1st- 
stiirting sites where students are typically a quarter to a half standard 
deviation behind the K-starting sites at the end of third grade. 



Table 6 . 1 and 6.2 



• * 

Table 6.1 
Sample Sizes and Sites 

— \ — K - - ■ . 

Follow Tlitough . - Non-Follow Through 

5th 6th ' 5th » 6th 



Flippin, .Arkansas 


24* 


. 18 


- 57 


^ 61 


E. St^ Louis, Illinois- 


A3 


47 


45 


"44 


Smithville, Tennessee 


* 

47 


48 


51 


47 


Uvalde, Texas ^ » 


117 


101 


^6 


101 


Dayton, Ohio 
« 


104 


33 


87 


47 


Tupelo, Mississippi \ 


^ ^ A6 


38 


34 


^ 25 


Flint, Michigan 


35* 




82 





Totals ' A26 285 ' 441 325 



^Started Follow Through in kindergarten. 
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Table 6.2 



Significance levels (one-tail test) fof Fifth and Sixth Grade; 
FT-NFT Cotsparisons with Covariate Adjustcenta 



t 




WRAT 












MAT 










FT 


FT 




N?T 


NFT 




^l 






^2 


WK 


R ' TR 


L 


Sp 


M 

Cosp 


H 

Con 


HPS 


In 


dCX 


-H- 
£< 


£ - 

.06 


0 , 

£> 
.15 


£- 

.06 


£< 
.05 


FllDDln 5th 


.06 
+ 


.07 
+ 


.09 


.06 


.05 
* + 


.03 .03 
+ 


• 10 
+ 


N.S. 


.13 


N.S. 


.12 


N.S. 


.001 


4 


4 


3 


3 


0 




N.S. 


N.S. 


N.S. 


N.S. 


.09 


•N.S.N.Se 


N.S. 


,C3 


N.S. 


N.S.- 


N.S. 


N.S. 


N.S. 


0 


0 


12 


I 


I 


S. St. touist 5th 










N.S. 


N.S.N*S. 


N.S. 


N.S. 


N.S. 


N.S. 


N.S. 


N.S. 


N.S. 


0 


0 


10 


0 


0 


£. St* Louis, 6th 




.01 

4- 


.10 .04 
+ + 


.08 
+ 


N.S. 


.06 
+ 


.10 
+ 


N,S. 


N.S. 


N.S. 


2 


4 


4 


0 


0 


Snithville, 5th 


.01 
+ 


.006 
+ 


N.S. 


N.S. 


N.S» 


.14 N.S. 
+ 


N.S. 


.05 

+ 


N.S. 


N.S. 


N.S. 


N.S* 


.15 

+ 


3 


2 


, 9 


0 


0 


Sfidthville^ 6th 


.001 


.001 
+ 


.001 
+ 


.001 
+ 


.001 


.001 .001 

+ - + 


♦001 

+ 


.006 


.004 
+ 


.001 
+ 


.001 
+ 


.001 
+ 


.002 
+ 


14 


0 


0 


0 


0 


Uvalde, s5ch 


.001 
+ 


.001 
+ 


.02 


.07 

+ 


K,S. 


.07 N.S. 


N.S. 


N.S. 


N.S. 


H.S. 


.01 
+ 


N.S« 


.15 
+ 


3 


2 


7 


I 


1 


Uvalde, 6th 


.002 


.002 


N.S. 


H.S^ 


-H^S. 


N.S. N.S. 


H.S. 


.05 


.04 


.02 
+ 


.07 
+ 


.02 
+ 


.14 

+ 


6 


2 


6 


0 


0 


-A 

Dayton » 5ch 


!ooi 
+ 


.001 
+ 


N.S. 
> 


.001 
+ 


.006 
+ 


.004 .003 
+ + 


.01 
+ 


.05 
+ 


.03 
+ 


.002 
+ 


.003 
+ 


.003 
+ 


.001 


13 


0 


- I 


0 


0 


Dayton, 6tb 


N.S. 


N.S. 


N.S. 


H.S. 


N.S. 


N.S. N.S. 


N.S. 


.05 


N.S. 


N.S. 


.10 


N.S. 


K«S. 


0 


. I 


12 


0 


1 


Tupelo, 5th 


.01 


.003 


N.S. 


N.S. 


.08 
+ 


N.S. .13 
















2 


2 . 


3 




0 


Tupelo, 6ch 


.005 
+ 


.001 


.07 
+ 


.05 


.15 


N.S. N.S. 
















3 


2 


2 


0 


0 


Flint, 5th 




N.S. 


N.S. N.S. 
















0 


0 


3 


0 


c 
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Exacxlnation of the patterns of significance by v^^xable show 
particularly strong results for ReadinR. On the WHAT Reading aeasures* 
14 of the 20 conparisons are highly signiftcant in favor of Follow 
Through* The children have never seen the tevel 2 WRAT r.cat before, 
although they have been tested before Level 1. On the MAT Reading 
measures, 11 out of 39 conparisons vcre significant In favor of Follow 
Through. Five other reading conparisons for the MAT or WRAT were close 
,to slgnifibance < .10). On WRAT Arithnetic, significant conparisons 
favoring Follow Through were found in only 4 of 20 instanced. The WRaT 
Level I Arithnetic test appears to be very insen^sitlve to the pxogran 
as later conparisons will show. On HAT Math a6s8UTes, JL2 of 40 cocj- 
parisons were significant. The rctaalning ceaaures showed 3 of l'^ 
significant f'-r Science, 2 of 10 for Un^u^ge, and 4 of 10 tor Spelling. 
One negative cooparison on the WRAT vas for WRAT Level I Arithnetic. 
In this case, the Level 2 Arithnetic test was alt»st slgnif ics'^^t in 
favor of Follow Through (£ < -07). The two other significant negative 
findings were on the MAT Spelling test. As we have noted before, Spelling 
was not a strong point of the progran in the early years. 

On HAT Hath, there Is a strong trend for HAT Problen Solving to 
show signiticant, or nearly signiiicant, ef f ects ,eve»j when other 
fceaaures of nath nvcabcr tacts r e going agains: Follow Through groups. 
This suggests that the problen solving approach taught in Dlstar 
Arlth!setic is naving long-cem effects. We would expect even stronger 
results to fihow when the stuf'ents get to algebra, since Dlstar fo»tusea 
row functions to prepixre studenis for what has been traditionally a 
difficult sublecc. 



on 
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* 

The findings In Science arc a nice surpria*^. And a strong cffcet. 
They can probably be attributed the Distar 3 Roarf£ng Progra:^ which 
uses a science content base to teach students to learn ncv nileu and to 
apply thea in their reading task«* 



HAGHITUDE OF EFFECTS BY SITE 

i 

, We vlll exaalne c^re carefuJlV the coeparability of the FT and 

HTT groups by site and illuatrate the general levels of pcrfonMince. 

With aaall aatspleOt uaing intact groups* it doca not tak6 tsuch in the vay 

of fl.i^ccion bias to produce good or bad results. To allcv'an examination 

of gro^ip conpa<ability» a Student Backgroutid For» vas filled out by 

persons hired at the aites to do this jjob (and/or by the FoUoy^rough 

parent vorkera). For each child the following Inlora^j^i^ vaa sought: 

!♦ Sex of student. - 
2« Level of cother*e education. 
3* Level of father's" education. 

4. ?fU3sber of siblihgo. 

5. Incooe status (above or belov Oeo guideline) v Other cvji<fencc 
accepted to define Lov-Incone vas eligibility' for free lijnches 
or eligiblUty for Aid to Dependent Children. 

6. White. Non--yhi^e thnic group status. ' 

7. Age, 

%ffc^u8e there vt?re Hoae differences between F.roups on these deaoures, 
up CO five- then wer*? x*>cd as covarlates to adjust cean outcoces for 
the dif terenctni. iTna available cotsputer prograa did not pemit the i^se 
c*f feore than covariatcs.? Education of parentn w^a not availablo in 

several slttN. Whcre^sal':5?*lng %iHt?. vu covari^i*** occurred (less than 
2 perci^nt of the c.iJ^.eo;, th*? groiip teean for this ^itv wa'3 inserted. A^<r-^ 
w*-ift not urifd a% .i covarlat**, but to drt«*r^ine WSAT standard ncorefi which 



i 

\ r ^ ' 

The cospariflos group for Flipplfl caae ttaa Coctcr, a ae«£by cosssuntty 



of aiftllar coapooltlon aad slightly lii^gcr In aitc. Mi but one fttudenc 
i?a4 VHlEc. Tabic 6.3 ahcvii the ecaao auJ acaatkrd deviaElona eft the back- 
gcoimd variables 'for the ?lippia cojBparlaous. the Follmf Tlicough ©othern 
.tend to have ssore education clwm the Hcti-Fdjiiotf Htsouab s<»thcrs. la the 
sixth grade, cHe HFT group hsj s»jrc girls, »>te sibo and the studimts »re 
£&ree ssoothe older. The variables oaed as covartstes are «arked viitb aa X 

Tsbic 6.3 ' 

the v^rlabie mflttcacing cov^srlauce adjutscssje^cs the* mf^i for the fifth 
grade co^sipoftrl^on« u«§ F^rh**r*«^ fducaUon^ Th0 iidjuo^eteni: tended to 
ioprwe thr* a^vm^tagi? of Follov ITirough ntvdmiB. Sex <of studc^i?:^} 
pji»r<*f%t4il f*d«ri*t Ion, nod inecsts^ fie^itun had s^^or influe^:c«s i^h^ 
a^jmt^i outmc^&n iot Che ai^cth grade cosrparieoofs. 

^Uifidinga. the WHAT* all calc^jldtt^^M were hm^4 on ^^^oten. 

The nom fc^t L**v«?i I «k* «K^t go i2 rt'^ra old, ijnd tho«i?'^f<ir 

Levul 2 do not g0 ibei^rv. Thin ctmi^^ ^dll<^«i^» rm^ scores v«?r<^ 

co?»vi¥ft«*d to ::;e-ftn gt^do c^*|uivdl€*nt/i luad th*:»?^ to percent: i less {ti^iifSi^, 4 
i^^n «g^* adjufttwnt). indicate tf^at the^tf figures ^ire based or? ^ 
dittiUti*:.t i^'^p'^t^ii^n bt%4Ci:4utc ihuu mt^d px^vi<^%mly Cor fetlU %i?»ed tor 
the HATis they h4ve tjt?eis plac<*d it% putmxthmm. To r^d^c^ th^ t^tnX 

&lm!( vlcH ^<-l«fCf4*d HAT nrotm^ Thi* 0i*p^riority of Che fifth grades 



Table 6.3 



Eackgrouad Variable for Fllppio FX-KFf Cosp&xU&m 
for Ftfch and Sixth Grades 



Pasta 1U6 



Sex (1 5» Haiie, 
- 2 * Feaale) 

Edac« Hi&tier 

School Gr'Jd.) 



Fifth 



FT 
24 



•J. 5 .51 1.42 .50 
4.62 1.06 

}.4fo 1,72 i.H K27 




Ci • I©v) I 
C2 » oot low) -^.i^ 1.6'. 



(I • White) 

(2 « N0o-ift>it#^;i .00 



Aga of Stti^ieoJ 
(years) 



.no i-cj 



4^ 



-00 



A. 89 .83 
6.50 l.M. 
3.22 1.80 



4.30 
3.92 



1.S9 
2,16 



1. 00 



Used m 
Covarlate (X) 



X 
X 

X 



t 



\ 



\ 
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PUppia Fifth Grade Pt-IIPT Coisparisons for UaEdjusced 
Test Scorojs; £ Values are for Covarisace Adjuatei Scores 



Total 
Reading 



tot^l 



Spelling Science 




*«BitseUne Is /irfoUrary 







FT 




FT 




FT 


HF7 


f 

> 












S 
















24 




6' 




64 




54 




?6 


56. 






90. ft 


9i.3 


/9.4 


79. S 


39.0 


82.8 


IK > 


1 li. ^ 




11:2 


10.5 


12.9 


^9.i 


9.0 




>.& 


6.2 


6.4 


S.8 


5^? 




6.Q 



Table 6.5 



Flippin Si«h Grade FT-KP^ Co©pariso)is for Oaadjuated 
Test Scoria; g Values are for Covsrl^nce Adjusted Scorcn 



Total 

Reading Aritncetlc R^adln^; 
044 r 



MAT 



Total 
Hath 



Spelling Uittn<o 



ERLC 




*WRAT »core«* are rav ur:orif^ 
^^BaneliM In arbitrary. 



PolXow Through chlldr^o i« on titt* ordtsr of oni?-half flt&nd/ard deviation 
on boch Reading «BBures dud in Gclanct** T'hi* gcore/j on WKAT Arit^ta^jtic 
are scraisgely and qulti^^ncoti^isttrnj: with the ebow gr^dt* level 
isc4?re« for HAT )tath. nn* repetition 6f thla pattern in awiny t^bUa to 
c^»e lea/j« ixr* to r^«4i?0Uon thi? /diidiry of the Arlth»4?tic t^&«i iir 

thlB lev**!. Tabli? ahwu that e^scepr f^^^r VRAT Arithmetic /ir4d r,j*elHng 
(our early negl^cr aK^^'r*), th^ Follow Througti children ^r^ Abov<^ grade 

fjor^ in th^> sixth gra^**» «fven thouR^t fhf dJfferi-n^ee do not nXi^k%\\t mxi i 

f 

fa*/ot Follow 7>*ro*igh. ^ 

covArlat/'«. Hi'/if* f'/ for Oi^ fHrh gr«4*' Follw Through chUdr^^n 

112. JiJ an*3 th4f for U^** ff^?3rp«irlaon grouj^ Thr^ rr/^rinnff^ iitJI u«f ^f*fif » 

for .jif fiTi^/j^^r-a in h^rk^ro^m^ '/ATinhl*:^ did not ^^hang*? tUn ?srigr^itMdf 

tnift dif f**r*-f»n**- r^r- probAblilt/ 3*fv**l for rh^ dlff^rf?rif^ v^iju 

(f>hiattlu^j IfJf- klndftrg.irt#*fi with ft r^^^t-f 10 ol ill./ fl^ - nu4 
i«ft thn /.ra^ii* win ^ a- fO of 1>0 % (n - ?1>, Ori«> ^igh* wifth 

«rj:u«* tr*/4t Uir- dlfl^r* ^^^^ if* ^srirJ '-r'l^nt*- nf*' dui^ to iO ill f i tn* 

but fhaf wf.uH nor i^ii^iaiu thi* f/»iiyr^ fo find 4i ^ f t^n^ur if. %|>^lilfi^f 
and H^th. Wi- ^^i.il^l no? 10 a tn^^tl^ti- ^tul^hn Ir w^d /iv^IUtl^ f 

both jprour.?* f.*ri iuJfi ro /^rt ->i . Tut^ drop lo fr^/e^. xh^ **nA ».f rhr- 
third gr^i^l** to fiffn K'^^id^ ^houh) not^d 

For th*' ^ilxrrj ^fA'j*- rhll<lr*-rj, taoari 10 1)/ - for n «fi4 lOV ?i 
for ?/rr, Tt»^ ^iffcri^n'^ w^/i r«^>i ^^Ignni^^uit or witho^r ^ /y^ar ) /t;»^ t- 

«d Ju^jf 0**nt n . Th^^i** f ijftvit % A ^ in^ ' Him i iuj/tj ^hii^Mw, i'f ^^*'*0 
trfith a cj^rtft V) of IHj /ini rhlr 1 ^^r.i'i** wffh n mr-,in 10 '<^f J>0. ' 

• O4 « IH) . Thr* dr'>;/ * n I'> in nivi^Mnr K'^ thf»t found (or thr fifth >/r*idtff^. 



The changed in rion/?-group ^t^ndinK frow chlrd to fifth ftrmi^ nra 
nhwn hnlou: 
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Arl f ^ti&i't Jf , wlil^'h flo fioff^^l xho/*? In i»rr^du'lriB a^rnriH^ly iw fJrMHiiK«* *f^»<^ 

ill* nittihut^ii ff/ rhf' fa*^f th/it rhi* Fiippio fhlHrt^o do oof for Hir cMiwe 
p^rr ro*t<* ff<^«» ho^&l^/^ whf*f«f thrro l« ntiotmaJ iMi/innifu ntlmulniUfU, 

Th^rf i/#*ri« oo K/T Tt»fir« lor rhit ttntmit niMth pjn^nxa wh^o Ihf/ Jrf» 
third ^'^^^ Arlr>mi»rjf /i oltsll^r prrf^nliir lo«^ waa f**ui,ii 

AH o->tt«*j fifth Kf«d«?i« (/0th ro JOrh f*itrr«ri>ni^)* On 

^.M^miinyi fUv iofi#* w«?i fr^m r hr H'^a'i j>*^f * r?n? 1 10 to t Uf»ih. 
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the growjio 4lffi?r on t^ntry Uh 11m? unfrronrtm of n li«H -wlnncJiird 

nlgoif If iinr . 
If, %i. Uidtt 

Thf f oL^(*/ir J noil fir»M*j/? for J. M, J.oulo * oi/ap tn*m iUv nnmp «<hiH»lw 

fihown fh«? brt»Vf«ro»m»l /fir f Iris , Unt it on ttdDf nt I on ol pnrrntfi Unti 
fiVftJl/ibJi^* tin Jfr>^ fh«ri hf»Jf *>! tliP pnmitu niiiJ nu ^nn nui nottij on n 
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The r**«alts give aoae support fcr poslrive Follow Through effects 
for the sixth grade group, and suRgcat that Foll.n/ Through «ktudentf> are 
loalng againdC the nona ftroup tn fourth and fifth gvade Reading and Hath. 
Earlier gains against the nora gioup are being lo^t, (Va viU be 
intcrcfSted in seeing whether tKis trend holds for K-startinp, cohorts in 
E- St* Louis who reach fitch $,r^iv chi9 yearO 
Sgithvillc 

Ffae cottpari^on :*cudents in SaUhvllle CiJee froo the saese schools 
aa the FoU'^w Through atudenta- The students in both »»"oup« are largely 
Whites, table shc*ws the background ceasurea and indicateti iho oaea 
a«fd as covarlate^. *a Mi^ Utth grad^ there sre oore HFT girls, an.^ .n 
the alxth gr^de »ore fl Klrl«. The s^FT (.^roupQ al»o hj^ve fever Lov-Incots^ 
parents Chan the ^jroupH. Tt^t^^ kinds inequalities can hs\^ ;iratsUc 
etferta cu Ci>&',i\x i JtoM . In covarlancc adjosttsenca* «ex# ni£»b4*? of 
^*tb$* and incoDc ^cacu^* f^ave t.^e tsosc 4^ftrct on the ouccottes. H^sc ol 
th^ adHi^nt^ent s Are siliecflv t.ivorlng follow Through. 

Table ^.10 hHcvh the f!:>Agt\it\id^^ etti^ccn tor fitth ^radem. -ml^ 
w1L\T Rehdlrc v lfE»arIv I »^*otii ^^V^ilcrv Through tn Che unrtdSustra data. Vub 
,1 P«i.f*tKt*nt HAT s^€*l*lnir « iKnl t leant . Th^ FT level 

r^t*rt<^n;an c ^r^ H.M HMh .'it th>^ n^Mon cedlan. evi>n though It l^* hc>hlnd 
cht* MKT ^t^^ri in t^e an^^dWHtd With c ova r lance ai3u^tet*nr Chi. 

situation revicne hovi^s^er t>»etsc cohort ) student at ^nov the folUwlnK 
changes ! r^^^s :^.;rJ[ ti r-^^ac -tifch Kfadrn: 
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BacVcgtourtd Variables for Ssi^chvlllo FT-HFf 
Comparisons for* Fifth ond Sixth Graden 
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The Kighri?^^ is most likely a car^over from the. I6we|^ 



/ r ^ S^^^^ iQ foy the 'Follow Thr.ou^li childfe^^ at the end. of fifths 

; - grade was 105-1 .and that for the ^con^ariison group* 102ff^^ the difference 
is^ hot "significant. These cohort 3 studeAts started if irst gr^^^ a 
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mean of; 100.1 (Ke 40) and left third igrade with a meaiHIQ pf 1X3.4. 
(H • 46) ; We would, ha^e to cbhclyde that th^se are legitimate comparlsOT^^ 
which, show a/superior^^ for Follow Through chVldren on WRA but 
losses against national norms during fourth and. fifth grades. 

Table 6.11 gives, the sixth gradte comparisons all of which significantly 
iavor "Foiloy Through cTiiH The cbvairiance adjustments operated to 

increase the' differences because the NFTVgroup had more education of 
parents, higher income, and. fewer sibiingk The Follow Through students 
also are signific'antly higher on SIT IQ before and after covariance adjust- 
iieht. Th^ me^n IQ for FT Jcids at the end of sixth grade Mas 108.6 (N - 48) 
■and ihat fot NFT,96.d (N *- 47). The Folljow Through children had a npan IQ 

• ■ ■ • ■ \ , . 

of 102.6 (N - 49) at the end of first grade and. 115 (N » 48) at the end of 
. ■- ' fable 6.11 3 
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.\ thus, itiiere is reason ip belifeve ;the differ^ncfc in IQ at 
Alx^h^irade iaaV te In part a function bf the prograa- Cohort: 2 was not 



^thixd: 



I 



%yfei (the ;MAt" in third gf^de, ' HoWeyer,W #e;PAT theM; Students 
ieft krfd grade with a »e^, percenHlC; tff.-S^^^^ in 
Arithketic. * they appear to^U naiiltalning >t>ei^^ reading -decoding- y 
.sktllsv. The loss- to Arithaltic is probAbJy M^^^^ a fimtti-on of fc^ 
■ WRA|:,Ct»^^^^^^^ is^tMcaUy pkoducirig low f i^h aW^ sixth 0^ 

. perfoiMnces relative to mi Math. The Sjaitl^^i^^^ 
-stfiii.a^^ * 

We conclude that there is a god^^possi^ld^^t^^^^^ thevcoSsparlsoh 
^ .5grqup, lyen .though more advantaged iii some w^8,,rh^ low periofming^ 

children,, in i ^ which inay be contribuVihg/tp s^^ of the. differehc^^^ in 




are 



favor of. Follow Through,. However i 
fromjFT isr excellent and cphsistep^ with 
thiiTv^ grade. The dverail lmpa<it Is that of a> positivie conclusion faw^^^ 



'Jhe sixth grad^^^ 
Dffmance on, ieaVln& 



' Foiibw; f hrp^ • 
Uvalde 
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Uvalde is a Texas commimity near the Mexican borders f The Follow 
Through-students; are mostly MexicanrAmef icans It wa? not possible to 
find cdffipar-isoh groups Uvalde who were. not in Follow ^rtirpUgh. However, 
comparispn groups were found dn thr^e consnunitles within :20-ml7as^of 
Uvalde ^whlch have students of "siniilar backgrounds (La Pryor,, Batesville, 
aiid Sabihal). Table 6.12 gives the information oh background variables.. 
Thi ^average education of all parents is below the 10th ^rade level. The 
grotlps /appear to be well matched on all background varial)les except that 

■ ■ ■ ■ ■ ■ ■ ■ ■ " ■ ■ ■ ■ 
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;the jflfj:V 8?«d?\^o^^^ Through students pa^^ Have slightly M^^ 
educa:tl^^^ levels, Coiopared to other FT and NFT stu^^^^ the tVaide 

' n^aiia^NFt are four months older In flfth grade and eight lion^^^ 

\ » ' • , - ' . . " V;-' 

V Ih: iikth grade. - The covafirace adjiistments prpdtfce only very minor • 

. . ;change8^ - . • . ' • - 
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fable ^,12 



^ The tesults for fifth grade show the /Istrstart^^^^^^^ 3' FT , 

.children vah^ad oh WRAT reading, ^They are behl^id slgn^^^^^ pn WHAT 



^ Arithmetic (Level 1 but not I-evel 2) which had showri/hb {sehs^^^^ 

^to .prograii. effects so fair* At.^^® ?ame time they are ahfad on^^W^^ 
; •(sl^if itMtly., |or Ifath Problem^oivlhg) ^ The g'ra^^ pf ;perr 

fprmance is on the Ipw side i The me^^^^^^ IQ for children (8^^^ 
.< at the end ofi fifth gra<ie is significantix higl\er <£ < .66l) than for 
,/ NET (79'6X, with or without cbviriance adju^^^^^ These FT 

/ :children enters grade with a jaean !(? of %T:.\ (N « &5) arid left 

third grade with a mean of '95.*8 (N «? 117) . In our l|dgmettt,, the net 
* difference iri IQ: still favoring the FT 'group coiiW be S p^rpgrim-effect. 
However, the evidence is open to q^^ „ 

^ Tiible, 6^i B . 
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Tabl^ 6.12 



^Backgraimd Variables forvUvalde F^-^NFT/Cotp^^^ 



I ^ - *^EdUc./Mbther; -\ . 
1 Schodii/Grad. ) 

l^/^r^t'-'lMtxc} 'Father -r-. , 



1^ 



Incojne' ieyel 

Ethttld Group : 

^ge- of /Student 
'(yeaj^^^ 



Fifth 



FT' 
il7 



^NFT 
'86 



Mean ' SD. Mean SD 



,;Sixth 



FT 



■., NFT 
101 



Mean SD Mean SD: 



i.53; .50 1.561 t 



2.86 1.61 2MI 1.25 1 

^ ■ - ■ ^ V ' ; 

2.86 1.6^ 2.aa; 1*21 ; 

6.21' 2.87 6.37 2:.58 ; 



1.13 .34 W65: 
06 1.95 



2.00. 

/ 

11; 5 



..22 



.22: 



•11.7 



;i.52 . .i50 i,5A 7.5Q 



• . . / 

.2.A,8 1.1b 2,50 1.35 

2,,A2 .1.29 2.36^ 1.11 

6;3| . 2v.6A 6X00 V 2. 



UOf .26 1.02 .iA 



li99 

12^; 



.10. ^2Vpo 
12^8 



00 



:lFse'd*-as\ . W 
Cpvariate, :XX)> 



X'.' 





4*C 




















- .J 





























4? ' '. 



. ' '■ ' ^ faWe 6.13' 

Uyal^de Fifth Grade FT-fJFf. Com^^^ Scores; 
' 2' Valiief are. for i^^^^ Adjtisted Sc^^^ . 
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Total Reading 
Total Math 
Spelling 



56 th 4.6 '116 

'39 th . 3.8 117 

31st 3.0 '117 

53ra 3.9 116 

40 th. 3.4 116. 



45th .6.0 '*117 

18 th 4^ ,in 

I6th°" 4.0 _ U7 
24th " 4.7. 'U5 
37th; ^ 116 



-11 5 1.5 
-^ll^ .6 



-15 
-9 

^ 3 



1.0 

- * .8 
1.7 



These charfges imply that perfortaance levels in Math are, not being 
maintained after Follow Through, while Spelling is being main,talned .and 
taiight. Thft low scores oh MA^ Reading are co nsist ent with the restriction 
on English vocabulary levels for children from Spanish , speaking homes, 
As we have pointed .cut elsewhere (and will again. in the Discussion^ 
chapter), the vocabulary skills' required , for performance oh MAT^'Reading, 
from th'§..Elementary Test Level on, .are skills which aire^r the most 
part not tauglft. in school, but in the home. The hi^h^r jelative performance 
on WRAT Reading is consistent with the initial/focus of Distar on , ^ 

decoding skills. The' students have^ learned; to decode worlds as well as 
middle-class students; they just. don't have the breadth of vocabulary 
comprehension assumed by the tests. ' % 

The sixth grade comparisons are shSwn in Table 6.14. Significant 
differences favoring Follow ThrdugK are prejsen^ for WRAT Reading measures 
^and most HAT measures except Total Reading. The low WRAT Arithmetic 
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scores look like, those we have'seeri b$:fore. The low- Total Reading pn the 

, " . ; ^ • ' • \ 1 ' ' ^ • ■ • - ^ ^ • ^ V ^ 

HkT Is consistent with our interpretation of the role home language , • 
training (vocabulary) /plays in^ learning the skills ^mcadured, by the test* 



favo 



A significant difference (£ •001> on the <51os8on is again found, ; 
" ^ . ' ^ ■ . — . ' . ' f 

prihg Follow Through stydents (84^9 versus 78 •SK These FT students 

entered FT ^with a me^n of.*91.4 (N « 57) and left third grade with a mean 

of 93;i5 (N »''92)^ Again, we ^re inclined, to interpret the difference at 

- • / ' ' . " . . . ■ ' ' • ^ 

siith :t|^.^^^ program effect, although ^this Is not proven. 

. The Cohort 2 FT student:^* left* third grade \?ith a WRAT Rerading 

* • * - * 

pftrcenttle of 66 (N « 920, "and Arithmetic percentile of 38 /(N; -'^^3), 



iy. the end df sixth ^grade, the mean percentiles have dropped* to 32 and 
14, respedtiVeiyi , . " * 

In summary, 'the Uvalde data show Follow Through strepg^ths in, reading 
jSLeco^iiig and the general competence ' measured bf , the SIT. For the sixlf^i . 
graders only, they also show st^ngths on a variety of MAT skills with 
the exception "of iTotal Reading.^ * * . ' ' r' 

' , ' . ^ " , Tahie 6*14 , ^ ' . 

\ . / • • 

•■ , ■ ■ 7 ■ 

Dayton. 

. >^ • 

The comparison students for Dayton came from a variety of schools 

within the city* Nearly all FT and NFT students are Blacks* The fifth 

1 ' ' 

graders appear to be quite comparatile on all variables including the 

' ■ i ^ * ' , . . ' ^' ' • 

partial information on parental education* The 'sixth grade background _ 

variables appear to favor the NFT group (many more girls, higher education 
of parents) . However, since education of parent could not- be used as a 
covariate .due to missing information, the actual adjustments we^re in- 
consequential. ^ 



Table 6 .15 



Page 167 



Table- 6.r4 



Uvalde Sixth Gradp FT-NFT Comparisons ' for Unadjusted Test Scores; 
,^ '£ Values, are for Coyariance Adjusted Scores 
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*WRAT scores are raw scores. 
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Backgroixnd Variables for Dayton FT^-NFT .Comparisons 
1 for Fifth and Sixth' Grades ^ 
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1.46 .50 1.49 "V-50 



(4.75 .46 5. 
(5i00 - .58 5. 



4.76 1.99 4. 



00 1.32) 
08 3^.38) 
M 2. -21 



1.15 .36 1.29 



,46 



2.00 00 1.91 .29 



1.36 .49 1.62 .49 

i\.86 .99 5.33 1.63) 

(5.09 1.22 5.24 1.03) 

4.59 2.47 5.00 2.36 

1.33 .48 1.36a .^9 . 

t 

2.00 00 1.91 • ^29 



11.2. 



12.1 



12.1 



.X • 



*Ba8ed on less than half, the parents. 
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* The fifth grade datd are shown in Table 6.16. Nearly all measures 
(except of course WRAT Arithmetic) favored FT students. The/dif ference 

/ r ^ ; • 

on VTRAT Reading is very impressive (more than one-half standard deviation). 



The othex: differences are on'^'the order of one-fourth standard deviation. 
The changes:, in. performance from third to fifth grade are as follows: 



^hird 



.%-tile G. E. 



N 



Fifth 



Z-tile^ E. 



Two-Year 
-Gain (LosaQ • 

%-tile G. E.- 



WRAT . 

Reading^. 

Arithmetic 
.^lAT 

Total jFleading 

Total \kath 

.I" 

Spelling 



6|th 4.7 
50th 



102 
102 



40 th 3;3\ • 102 
55 th 4.0 \ 102^ 

40th 3.4 ; -92 

^ 1. 



6is1: 


(7.2) 


93 


-6 


2.5 


ig-th 


V(A.2) 


95 


-31 . 


.2 


4 

20rh 


4.4 


104 


-20 


1.4 


19th 


4.5 


102 


-36 


-.7 


'34 th 


5.0^ 


104 


-6 


1.6 



Table 6.16 



•/ 



V 

.lio IQ data are. aj^ailabie lor the Dayton fifth and sixth graders. 

However, the fifth graders left third grade with a mean SIT IQ of 101.0 

(N =^103) anS the sixth graders with a me^n of 112 (N « 33 ) . The 

comparisons from third to fifth grade on the WRAT and MAT imply that . — - 

j ' . ' ^ / - 

spelling skills are being taught and reading decoding skills are being 

maintained. However, there are definite losses against the norm group 

on reading comprehension skills and math. In spite of these backward 

mov^s, the FT children are superior to th^^ NFT children on 13 out of 14 



measures < 

\ 

\ 
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84- 



77 

69 

60 



^ 50 



=3 
O 

Of 

c= 
o 

.c/> 



o 



40* 
31- 

23- 
16 



11 
7 



Table 6.16 



Dayton Fifth Grade FT-NFT Comparisons for Unadjusted Test S^res;^ y 
£ Values are for Cbvariance Adjusted Scores * 



WRAT Level 1 
A 



TR 



TM 



MAT 
Sp 



Sci 



Ji ' 



IS 



i 



.001 



,N.S. 



V 



cr -3* 



.003 




.003 ^ 



.05- 



r 



■-3 



r IS 



11 



'-'''1 ' 



.00: 



4 
















i 


■4, 








GROUP 


FT 


. NFT 


FT 


NFT 


Ff 


NFT 


FT 


NFT 


ft"" 




_ tt 


NFT 


POST 


5 


5 


5 


5 


5 


5 


5 


\5 




5 


5 


'5 


N 


93 


55 


95 


59* 


104 

t 


87 ^ 


102 


83 


104 


• .85" 


102 


^5 


%..tile 


(61) 


(27) 


(19 


i:i8) 


20 


. 16 


19 


12 


:^4. 


27 


20 


13 


S. S.* 


71.8 


63.-6 


31.2 . 


31.0 


68. u 


64.9 


77.3 


74.6' 


74.0 


71.4 

< 


73.2 


6^:5 


S. D*fl4.6 


14.1 


3.5 


3.5 


10.3 


10.0 


9.0 


9.1 


10.2 


10.5 


9.4 


7.5 


fi. E. 


(7.2) 


(5.6) 


(4.2) 


(4.2) 


4.4 


4.0 


4.5 


4.3 


5.0 


'4.7 


4.5 


3.9 



*WRAT scores are raw scores. 
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The. sixth grade comparisons show a ninnber of trends favoring 
FT students such as WRAT Reading, Math Prdfclem Solving, and Science. 
How^^er, no differences are significant after covariance adjustments. 
If full data on parental education were available, the result might 
be different. "At th^ end of third grade, these cohort 2 children were/ 
at. the 69th ^percentile on ^mAT Reading (N 33)» _ _ 
On WRAT Arithmetic, they were at the 55th percentile. There are sharp 
"^^^ses on both WRAT Arithmetic and MAT Math against the norm group. 

Spelling, is also very deficient for the sixth grade Follow Through group. 

'..I " - ' . ■ ■ ^ ■ . 

Th'is was nat the case at tbJ&> end of third grade (WRAT Spelling 73rd 

percentlle^ersus 13th petcentdULo: oji the MAT at sixth grade) . A . 

Strong possibility is that these children are not being adequat^ely taught 

in the upper elementary grades. 'These results ark also perplexing in view 

o^^he^mean IQ of 112 for these students at the end of third grade. 

^ Ta&le 6vl7 » ' 
J^upelo , ^ 

.The comparison children in Tupelo eame from a variety of city and 

county schools. The'Hjackground variables are summarized in Table 6.18. 

T^e control group has more Whites than the FT groups which are mostly 

Blacks. There appears to be a lower educational lev^l for tWe sixth 

grade NFfS^atents and the fifth grad^ FT parents. However, these ^ 

variables could not be used, in,' the cavariance analysis because of the 

large amount of missing data. The fifth grade NFT group also has more 

boys. The covariance adjustments that wer^ made do not materially effect 

the^ group .differences except on fifth grade MAT Jo.tal Reading, where the 

adjust^Qent leads to a difference t^^d favoringj Follow Through. 




1?age 1T2 j 

I ■ • 



Table 6.17 



Dayton Sixth Grade FT-NFT Comparison ior Unadjusted Test Scores; 
* £ Values are for Covariance-AdjAist^d Scores 



m 

o 
CO - 

o 

is 



84- 
77- 
69 

iso 



4 



40 

31 




TR 



TM 



MAT 



aci 




GROUP- 


-• .FT 


NFT 


FT 


NFT 


•POST 


e 


6 


6 


6 


N 


18 




20 


25 


%-tile 


(50) 


(27) 


(13) 


(12) 


• S.S:* 


75.1 


69.4 


32.6 


32.7 


" S. D.'*' 


14.4 


15. ,2 


2.9 


4.0 



erJc 



^.£.^^.0X6.0) 

*WRA' 



(4.5)(4.6) 



FT 
6 
33 



Sft 

6 



47 



22 - 22 
74.3 73-.9 



7.8 
5.1 



14.0 
5.2 



FT 
6 

32 
*'19 



NFT 
6 
45 
17 



83.5' 82.5 
8.1 10.2 



,T scores are -raw scores. 



5.2 



5.2 



FT ■ NFT 
6 6 



32 
13 



45 
.25 



71.6 77.4 
11.. 0 11.8 
.4.7 5.^ 



FT 
6 

33 
23 



NFT 
6 

■ 45 
21 



78.4 77.3 
8.2 9.9 
5:2 5.0 
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Tables 6.19 and'6.20 show the Follow Through childrea do significantly 

' , \ - \ ' . , '* ' • 

better on VJRAT Readiiig and a trend for a better performance on California 
^Achievement Test Reading (converted to MAT equivalents). FT children 

. ^ ' ^' ; i ' , 

alsd. are better' on WHAT Arithmetic at the sixth gVade, but we' do not 
trust 'this. test. The mean SIT IQ's of thfe fifth graders are a little" , > 
lower. th.\n ,their comparison groups (88.9 versus 92.3). The mean IQ of theise 
cohort 3 children at the end of third grade wais (N » 42). The t£ sixth 

' " s ' - . ' / ' ^ ! 

grat'ders have significantly higher IQ*s than their comparison group (92i5 
to 84.4)^' This, group lef t cthird grade with a m^ah IQ of 100.9 (N « 5U;» 
Again, it is "difficult to interpret this difference, but ii^ our judgment 
the'NFT comparison group for sixth .grade is a lower, performing ^ro^^ 
This conclusion is supported by the lower. education levels of the parents 
"on whom we have data. Thus, we would tend to discount the sixth grade 
data, but not the fifth grade data. . 



Tables 6.18, 6.19 and 6.20 
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Table 6 .IS' - 

BackgroundQTa^i^bles for Tupelo FT-NFT Comparisons 
^ ^^r Fifth and Sixth* Grades 



Sex (l.« Male, 
2 » Female) 



I5duc?Mother* 
v(4 - High 
School Grrad*) 

Educ» Father* 

No* of Sib£y . 

Inccne Level 
(i »■ low, 
2 - not low) 

Ethnic, Group 
(1 - White, ' 
2 - Non-White) 

Age of Student, 
(years) 



Fifth 



FT 



NFT . 
46 / ■ 34' . 
Mean SD Mean SD 



1.48 



.50 1*35. 



.48 



(3.22 1.29 3.58 1.62). 

(3.56 1.00 4.12 1.15) 

4;41 2.22 5.12 2.29 

1.05 ...23 1.03 .17 

1.93 .26 1.53 .50. 



11.2 



11.2 



Sixth 
FT NFT 
" 38' \25 
Mean SD Hean SD 



^ 1.-55 - .50 ' 1.48 .51 

(3.97 1.55 2.20 1.30) 

(3.89 1.17 2.60 .55) 

4.53 2.06 4.60 1.52 

lilO .31 1.00 00 

1.91 .29 1.'68 .47 



12.0 12.4 



Usecl.~a8=, " >• 
Covirlite (X): 



X 



' X 



*Ba8ed on leBS than half the parents. 
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Table 6.19 



Tupelo Fifth Grade FT-NFT Comparison for Unadjusted Test Scores; 
£ Values are for^Covariance Adjusted Scores - 




















. GROUP FT 


NFT 


. FT- 


1>IFT 


• FT 


NFT.' 


• •'v 


. ' POST 5 


5 


5 


5 


^5 


5 ' 




N ■ 56 


34 


56 


3A 


56 


33 


i 


•%-tile (-^2) 


(27)' 


(21) 


(2-3) 


18 




s 


.s. s^;* 67.3 


63.3 


31.5 


32.5 


66.8 


,.67.0 




S.O.. 12.3" 


11,9 


2.8 


3.3 


10.6 


n.k 




'6.1. ^ (5.6) (A. 9) . 


CA.4)(4.6> 


4/3 


4.3 



/ ■ 



4) r *«I^RAT scores are raw scores.. 
1?ERXC *^CAT-. converted to MAT equivalents. 
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CO 



^ O 

c: 
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84- 
60 

> 

40 

31 
23 

11 

, 7 
4 



Table 6.20 



, Tupelo Sixth Grade FT-NFT Comparison for Unadjusted Test Scores; 
£ Val.ues are' for Covariahce Adjusted Scores 



WRAT Level 1 
R ' 'a 



MAT** 
TR 



GROUP ' FT NFT 

POST 6 6 

N -38 25 

%-tile (AM (13). 

-S, S.* 72.4 64.2 

,li"P.' ' 8 10.6 

8. E. .(6.6) (5.0) 



FT 
6 

38 



NFT 

. 6' 
25 



(17)' (10) 

33.7 32.6 

4,2 3.1^ 

C4.9) (4.6) 



i 










































































t 




































\ •: 




■ 




, 1 


/ 












1 


















-I 








1 










- 

Ji 






















■? = 






















.1 


























jT" 1 
















i 








= .005 




.07 ; 





















FT 
6 
37 
36 



'NFT 

< 

6 

/24 
28- 



73.8 71.2 
'9.7 9.0 
(5.0) (4.6) 



:ERJ.C 



*WRAT scores are raw scores. 
**CAT converted xo MAT equivalents. 
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The Follow Through children, showed changes in hortt group standing 



.-'ft 



froin third; grade as follows: 



Third 



Fifth 



Two-Year Gain 



%-tile- G. E. N %-tile G. E. N 'Z-tile G, E. 




4.4- 
3.9 

y 

'2.9 



42 
42 

42 



(42nd) (5.6) 56 - 24 . 
(2ist) (4.4) 56 ^32 



18th 4.3. 56 ■ -10 



1.2 



Reading. 
Arithmetic 



4.4 V 



(4'4tii) , ^^6.6) 38 - 24 \.i 



3.6 



(17th) (4.9) 38. t29 1. 



While the WK^T Arithmetic data should not be taken literally,, the general 
* trend of the. data support a loss Igalnst the norm group after leaving' 
Follow Through ; Reading decoding skills are maintained,. however. 

Flint . . , • * ' ; • 

The comparisons for Flint are limited to three MAT Reading meafsureis. 
They show no significant differences/ The backgroimd variables presented 
in Table 6.21 show that information on parent education is lacking and 
that more than half of the comparison children are Non-Low Income, whereas 
only one FT student is. In our judgment these comparisons aire not . 
. valid for judging the program in Flint. There are farther complications 

■ - ■ ■ V' ' . ' ■ 

due to the fact that most Titl6 1 children In Flint were in Distar programs 

^ ■ ' • ' \ " « ' 

from the beginning of Follow Through. ' \ W 

^ \ Looking at the ^progress of fhe K-starting cohort 2 children from 

the end of third grade on MAT Total Reading, we see a drop from the 34th 
percentile (N « 41) to the 22nd percentile ( N - 35). The two-year gain 



■fi: 



' TDJr grade equivalents w^stfpm 3.14 tq^4j.35^ 



\ 
\ 



' « * , . Table 6,21 ^ 

Background Variables for Flint PT-NFT Comparisons 
•for Fifth Grade - . * 
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N 



Sex (1 - liale, ^ 
. ' .2 - Fc-ale) 

No. of Slbs 

Inccme Level - • 
. (i « low, 
2 ".not low) 

Ethnic Group • 
(l - White, 
2 - Non-White) . 

Age of Student 
(years) 



Fifth 



■ FT 

- 35 • 
Mean . SD 



" NFT 
82 

Mean 'SD> 



1.51 .51 l.,57 ■ .55 

4.11 1.59 4.65 2.18 

1.06 .24 1.57 .55 

1.94 ,24 1.77 .42 



11 -.0 



11.2 



ERIC 



•A 



2h: 



> 
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Todd County 

Although not a part of the controlled /comparisons /a sample. of 
Todd ,Couhty fifth and sixth graders was also tested* Since we have 
third 'grade MAT data pnly on the fifth ^g^aders, we will present only 
that- data so that gains from third grace can be examiiied. / 



' \/ ' « " ^ , Two-Year 
. 2hirdl__ ^ ™--.-Ii^._,„:_GM!^^ 

'/.-.tile G. i;' i'n %-tile G. E, ♦ N • %-tile G. E. 



Total Reading 


42nd 




•i43. 




^ '♦7 . 


44 


.,-15- 


1-.7 


















Total Math 


44th 


43 




4.7 


*44 . 

> 


. - 24- 


1.1 


Spelling / 




■3,9 

' 1 

f 


■43 


-3^8th'' 


.5. '2 ■ 


44 


- 14 


1.4 


" / • * 




/ * ■ 















The results show .some loss against^h4 norm' group,, , but not as 
drastic as for other places. Some reasonable gains are made on IJAT 
Reading after Folloy'4hrough and on Spelling. . " 



\ 
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Sunmary 



The follow?Tup study of more than 700 fifth and sixth graders," along 

" ^ ■ ■' ^ ■ " • ■ . ' 

with, comparison grccps, clearly .showed measurable persisting effects ^of 
, the largely S^^year E7B Follow Through programs,* The significant effects 
rrwefe particularly strong in reading on both the MAT and the WRAT, In 
, mathematics, the MAT showed a number of significant effects, especially 

for Problem, Solving r Slgnific'ant Foli&w Through effects were also found 
'on .MAT Science, . . . • 

^, Examination of Ttfthe equality of the giroups^ on background variables 

and^ the magnitude of effects by site revealed several Important points. 

K few^of the compurisons appear' ^to be invalid because the Follow Through 
,and comparison groups, were not adequately matched. However the ovejr-* • 
^all effects are not changed by this. It .appears t;hat some spelling Is 

taught .a'f ter tuird grade which helps; to keep percentiles on spelling 

close to where they were at the end of Follow Through. Reading decoding 

skills also tend to be maintained. However,^ large losses against the 

' % * . ' ' ' 

.norm group are apparent in math for most sites. The ipmlication is that 
effective teaching of math at the upper elementary levels in our Follow 
Through site^^s is not occurring. Status on the MAT reading comprehension 

« 

measure at the end ot fifih grade is typically low in the Non-White 
communities (Tupelo, -Uvalde, Dayton, E. St. Loui^) for the cohorts 
studied In several cases, this measure was 'also low at the end of 

third grade; in others there have been sizeable losses against , the norm 

' * ' " ^^.^ . ' ' ^ ' /\ 

group. We expect futuye cohorts to do better on this measure, especiallyr^ 

those. starting the program in kindergarten. * « / 

r- " ' . ' \l 

*Ih, 1976 this study was repeated with 600 Follow Through firth and sixtjx 
graders and -600 comparison students in six sites. The resul+. was near'iy 
identical. , ' • 
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* « , ^ .1 

To our way of thinking, the data support once again the conclusion 
drawn from Head .iStart data that gave birth to Follow Through. Gains made 
'through better instruction in Head Start \,are lost when the children were 
put throijgff^into an educational system t'aat was not effective. There is 
clearly an implication and a need to study., the effects of continuing a 
-Follow Through type program until adult competencies are mastered. 
^ . . In OUT' view the basic . ? 
d^^T^iency in performa nce*^ jp^ children of^ povex^ty is in the size of 

their voj^ulary .(language concepts). To a large extent, language, 
concepts arc^ taught at home,^not school^ f or, middlerclass kids. By 
the end 'hf thirdfgradei the typical home and school have taught an 
*bral vocabulary of from 5000 ta 8000 words, and a .reading vocabulary of 
ZOOD^words. The children are then thrust upon testbooks (and tests 
pf read^^^ comprehension) where comprehension of the full adult 

vocabulary is expected. (Thomdike-Lorge estimate that the typical 

* \ " 
high schoojS\senior has a reading vocabulary of 15,000 concepts). It 

is not surprising that children of poverty are more likely to flounder 

at this time. As we see it, the E-B* programs have overcome much of 

the g^p in language comprehension, but in Ist-^starting sites it might 

take two toore years of extensive language instruction to * 

"lick the problem," and in K-starting sites another year.. An alternative 

solution is to change educational systems so that there is systematic 

languag^ instruction in the schools (K to 12) which progressively builds 

an adult reading vocabulary of 15,000 words year-by-year,' and does not 

leave the instruction primarily up to parents, TV, and whatever. We are 

■ .^-^ ' 

developing approaches to dealing with this problem in the near future, 

; f ■ * • ■ " 
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CHAPTER 7 \ 



COMPARISON OF PROGRESS IN RELATION TO IQ 

i 

^The Low IQ Studv 
^ ! ^ 

The data for all children in our main file with IQ*s on the 
Slosson Intelligence Test under 80 at. entry to the program were com- 
pared with that for our total group (All). . The data were'^ analyzed 

c 

separately foi^ Prj2-K to Posf-K and Poat-K to Post-rS, and for Pre-1 to 
Post-1 and Post-^1 to Pos|2^3. The data show that Low-IQ children (mean 
73 at pretest and ranging, as low as 50) gain more than a ^rear on WRAT 

^Reading for each year of •iti^st ruction.* On the averages the gain is 
approxj(jnately 1.2 grade levels each .year, while^ the gain for' the All 

'group averages 1.35 grade levels each year. 



Figure 7.1 



The gains on WRAT Arithmetic are shovm in Figure 7.2. The results 
are very similar. K-starting Low IQ students actually make larger gains 
in ki idergarten than the All group and show average gains over four years 
of .95 grade equivalents. The average gain over four years for the 

/ — 

All group is 1.00. The Ist-startin^ Low IQ students nearly match the gains 
dfi. All group during the first year^and over three years. The average gain 
f 5r the Low IQ group is 1.04 gra<^4 equivalents^ and for tha All groiip 1.07. 

Figure 7.2 



210 



1^ . 

. Pre K- Post K 
Low iQ (107) 
All (2118) 



Post K - Post 3 
low IQ ( 



-1 



Pre .1- Post 1 I 
LqwI|Q(305j 

All (2113) 



Post 1 - Post 3 
Low IQ (408) 

. All (3333) 



-1 



PaKe 183 



[W;?999<1 1.24 gain 



K- Starting 



>6<XX>OCl l.33 gain 



\ \ 

-1 K- 


—4- ^ 1 

1 J 2 3 


4 ^ • 

4 5 






K - Starting 

3.32 gain 






>O660><xxxxxxxyxyyyyyi 4.04 a^in 


~ ! 


^ i -r+- 


\ : h-^ 



l^^^g^^ 1.33 gain ' 



1st - Starting 



I9S666566<| ' 1-72 gain ^- 



2 



3 



5 



' . 1st - Starting 

2.52 gain 



^^^^^^ 2.74 gain 



1 



2 



3 



WRAT Reading Grade Norm 



Figure 7.1 Reading gains of low IQ disadvantaged children. (Shaded area. 

Indicates gain for the time period shown to the left of each c"har£.) 

Low IQ =■ IQ of 80 or less in Engelmann-Becker Follow Through Program. 
All = All children in Engelmann-Becker Follow Thrcjugh Program.. 

'i>. 



2r; 



4 



Pa«e 181* 



Pre K • Post K 
Low IQ (134) 

A!ri2301) 

Post K^- Post 3 
Low Id (300) 

.All (2088) 



[><XXXX>^ 1.43 gain 



K - Stafting 



Pre 1 - Post 1 

iowTQT406) 
: Al)(3310) 



Post 1 - Post 3 
Low la (40). 
All (1376) 



565666^ 1.29 gain 



2.39 gain 



K- Starting 



KXXXXXXXXXXXXl 2.70 gain 



5: 



i^^^^^^ 1.22 gain 



'•.4st- starting 



l^^S 1.28 gain 



3 



T- 

5 



/ 



IWSMM^^ 1.91 gain 



1st - Starting 



>6666<r>66<5^ 1^93 gain 



- — ^1 1 H — -■ 1 1- 

-1 K . 1 2 3 4 
WffAT Arithmetic Grade Norm 




Figure J. 2* Arithmetic gains of low IQ disadvantaged cbildVeji. (Shaded 
area indicates gain for the time period shown to the left of 
each chart.) 

- * 

Low IQ « IQ of 80 or lesS/in ^rigelmann-Becker Follow Through Model. 

All = All children in Engeimanp-Beckqr Follow Through Model* ^ 
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V 



Mean Gains in Standard Scores for .WRAT Reading and Arithmetic 



Pre k - ■'Post K 



JP-osfK r Post 3rd 



Reading 
£ow-IQ 
ALL- 



Mean Gain 
24.9 
24.5 




SD Gain . 
20.6 
18.0 V 

'15", 8 
16.2 



Mean 6a in 
11.2 
-8.5 

.72- 
-5.^ 



SD Ga>in 
17.7 
18.0 



10.0 
11.6 



' . • ' Pre 


1st - Post 1st 




Post Ist - 


PoBt 3rd 


Reading 


Mean Gain 


SD 


Mean Gain 


SD Gain 










Low IQ 


14.-5 




9.0 


6.9 

-* 


ALL 

1 


^9.4 


15.8 ' 


12.3 


.16.0 


Arithmetic 


* 








Xpw IQ 


11.6 _ , 


11.6 . 




8.7 


Ahh 


11:0 


11.3" 


0.0 


9.8, -0 



The mean gains in standard scores are given in Table 7*1. 

J • ^ 

Except for the Ist-starting group in reading, the gains for the Low IQ 
Groupware equal to or better than those for the AU. Group. 

An examination of, the number of lessons taught each year to the 
v^tous groups shows that Low IQ children are taught 30 fewer Dlstar 
jlessons 4n Kindergarten (160 lessons is avei^age) and 25 fewer In first 
Igrade. After that the differential drops to less than 15 lessons. 

;i?here is an implication tha^t what a child can learn is less a functiort of 

^ . } ' ' 

IQ and more a function of method of Inst ruction .^^^ 

2iU 



: TaH^ 7.1 shows the* IQ gains by groups used in the analysis* 
Statistical regression effects would make the Pre-K to Post~K and 
Pre-1' to Post-1 gains for the Low IQ groups *y eater than the true gains. 
The average regression can be predicted by multiplying the deviation of 
the Low IQ group* from 100 by .92 (reliability of SIT IQ). For the K- ' 
startStng group whose mean is 73, this procedure- gives an estimated true 
deviatlQu score of (73 - 100) x .92 « -24-.8 or an estimated true mean 
IQ of 75\.2» The K-star ting, and Ist-startfng ALL groups both show net 
IQ gains Pre^-K to Po6t~3 of about '8 ppiptfff No' regression effects 
influence the&feNga}ns» Thus,, we would estimate the true gains fpr the 
Low 10 students toVbe somewhere between 8 and' 14 points'* 
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• Table 7.2 - - 

Slossbri IQ GA-INS for Low ±Q and ALL Groups 

V 

€ / 



Pre K - Post K • . Pre • 


• 


post 


Gain" 


Adjusted 
Gain 


SD Gain 


N 


Low 10 ■ 73.0 


(75.2)* 


93.8 


20.8 


(17.6)** 


12-. 1 


142 


^ ALL. 103.3 




112.8- 


9.'5 




14.2" 


22^0. 


Post K - Post 3 - "IPre 

• ^ 




Post 


Gain 


Adjusted 
Gain . 


SD iSain 




Low 10 80.3 


(81.8)* 


87.8' 


T.5 


(7l0)** 


15.0 


38 


"XlL 109. 3 




107.9 


-1.4'. 




14.0 


1305 


Pre 1 T Post 1 . Pre 




Post 


Gain 


Adjusted 
* Gain 


SD Galn^ 


N 


Low 10 72.5 

1 ^ ' 




85^5 


13.0 


(10.8)** 


L0.8 


319 


ALL . • 94.7 




100. i 


5.5 




12.0 


2113 


Post 1 - Post 3 Pife 




■Post 


Gain 


Adj us ted 
Gain 


SD Gain 


N 


Xow IQ 78.7 


(80.4)* 


83.4 


4.7 


(3.0)** 


12.9 


387 


ALL ■ -99.^ 




101.6 


2.5 




13.4 


3081 


* Estimated true score means. 












** Estimated true gain. 
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/ . . * 

-* ^ 

' ' Relation to Entry iQ to Academlc/Perf ormance 

A;» analysis was performed 'to s.ee if children with different entrjj 
SIT IQ's benefit more or less from the program. The general result is 
illustrated in Figures 7*3 and 7.4. Level of performance in ^standard 
scores or grade equivalents shows a systematic increase with higher 
starting IQ's. For the mosi^.part, however, the year-to-year gainc do 
not syst^ematically increase as starting IQ increases. Children with 
higher entry IQ's start higher on mo.st^achicvement measures (reading, 
arithmetic, spelling) and stay higher, but do not usually^ain propor- 
tionately more. • * 
- . \ \ * ' 

. Figures 7.3 and 7.4 about here 



There arc cwo exceptions to this conclusion which ar€^ illustrated 
In Table 7.3 and 7.4.- In going from the Primary II MAT Total Reading (end 
pf second grade) to the Elementary. Level (end of third grade), a clear 

V 

.relationship between amount of gain and IQ blocjcs is apuarent. This 
finding would be expected if general language development (yhich is 
fostered to a large extent outside of school) plays a key role in both, 
the IQ measure and the MAT Elementary Reading Test. This finding is 
consistent with our interpretations of other f ilndings on the MAT. Elera- 
entary Reading test, and will be treated in more detail in the discussion. 
The other -exception is the first-year gain (Kindergarten for K-Startfing 
and First Grade for First-Starting) on WRAT Reading. Children with 
higher entry IQ's gain more in reading decoding skills in their first 
year of instruction. This may be because their better language skills 
give them an advantage in beginning instruction, (e.g. some children do 
not have to learn the language of instruction before learning to iecode) . 
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> Figure 7*3 ' 

WRAT Reading Mean Standard Scores for Ist^Starting 
. Children by Entry SIT Ip Blocks 
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^ ' "F^igure 7;4 

MAT Towl Heading Mean Grade Equivalent Scores 
ifor Ist-Starting Children- by Entry SIT IQ Blocks 
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• Table 7.3; . 

WRAT* Reading Standard Score. Gaitis by Grade Leyel for 
. Ist-Starting Students by. SIT TQ Blockis^ 





' First Grade 


Second Grade 


Third Grade ^ 


IQ Block 


Mean 
Gain 


• . N ■ 


Mean 
Gain 




'Mean* 

'Gain 


N; 


. 70- and 
below 


Id* 1 


AO 


Q A 


JO 




« 

'42 




16.9 


366 


8.1 


426 


-1.4 


400 


91^100 


19.8 


528 


5v2 


373' 


-f .2 • 


353 >. 


lOl-llO" 


21.1 


214 


5.3 


■ 241 • ■. 


-.4 


225 


111-136 ' 


25,3 . 


161 


3.9 


\93 


\1.1 


■167 " 


l3l and 
above 


24. 0 


25 


8.8 


28 


5.9 


17 , 


Average 


. 19.8 


1143 


6.2. 


1316 


- .62 


1204 



MAT Total Reading Standard .Score Gains by Grade Level for 
Ist-Starting Students by SIT IQ Blocks 





Second 


Grade 


Third 


Grade 


IQ Block 


Mean 
Gain 


N 


Mean 
Gain 


N 


70 and 
below 


13.8 


14 


3.3 


Z3 


7i-90 


14.2 


122 


4.1 


192 


91-100 


13.3 


106 V 


4.6- 


198 


101-110 , 


13.9 


77 


6.3 


107 


111-130 


14.5 


44 


6.8 


58 


131 and' 
above 


"12.4 


14 




C4) 



Average 



13.8 37/ 



22 



4.9 582 
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Table 7.4 

, \ \ *r WRAT Reading Standard Score Gains by Grade Level for 

: \\ - ^Starting Students by V:t IQ Blocks • . : , 





• Kindergarten 


First 


Gfade\ 


Second Srade 


Third Grade 




Mean 
Gain 


■> 

N 


Mean 
bitin 




^ Mean 
Gain 


N 


Mean-^ 
Gain 


. " N " . 


'71-90 


19.5 


18 


^V9 " 

\ 


39, 


-2.6 


43 


-3.5 


42 


91-100 


15.8 


52 


, 9.3^ 




-3.7 


100 " 


-.39 


'91 




19.6 


64 


11.4 


124 , 


^4.2 


133 • " 


2.86 


'123 


lli:-130 


■ 24.2 


61 


9i'3 


142 ■ 


-6.1 ' 


i49 


1. 


138 • 


1>1 and 
above ■ 


29.2 


12 


7.4 


22 




22 


-2.52- ■ 


23, . 


Average 


20.5 


209 


10.1 


429 


-4.4 


- 450 


1-.36 . 


420 



MAT Total Reading Standard Store 'Gains by Grade Level for 
- K-Starting Students by SIT IQ Blocks 



IQ Blocks 


Second 

Mean 
Gain 


Grade 
N 


Third 

Mean 
Gain 


Grade 
N 


71-90 


9.0 


15 


' 2.3 


41 


91-100 


9.7 


37 


3.7 


82 • 


lOlVllO 


7.6 . 


49 


5.0 


115 


111-130 


8.7 


76 


6.8 


128 


131 and 
above 


9.4 


14 


6.1 


23 


Average 


8.7 


192 


' 5.1 


392 
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This evidence demonstrates^ that the Direct Instruction Model is 
clearly effective with Low IQ children. This is the group who h'aye 
been. hardest to teach ihT^fie past 'With good instruction that builds 
-,6r; their ^ntry skills in a systematic .way. Low IQ children c2n be 
taught to be smarter and more capable. 

vTbe evidence also demonstrates that entry IQ is not a major 
determinant of academic gains in the direct-instruction programs. 
Where some relationships to. entry IQ's are found, they Seem closely 
tied to the role .of general language competency in what is being 

.meas^tred (Elementary MAX) or to the language skills necessary to 

' ' " ' ■ ' 

facilitate beginning instruction. 
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CHAPTER 8 



FORMATIVE EVALUATION OF DIRECT INSTRUCTION ' 
* Douglas, W.I Carnine ^ j. 

Direct Instruction's effectiveness, as indicated bV the earlier 

. . . : ■/, . 

chapters in this report / has also been documented in qtKer Follow 

* * ' ■ ' 1 2 3 . // 

, Through research * * arid in summaries of Distar research conducted 

" t ■ 4 5 r- ' " ^ 

in non~Foll<sw Thraugh s^ettings. * Although the data indicat.e, that 
direct Instnictlon is effective, questions concerning direct instruc*- 
tlon*s functiouux components remain. The Direct instruction Program's 



components include teaching techniques, procedures for teacher training, 
and the Distar curriculum, to naiee a few. Identifying the functional 
components are important for both developing a technology of education 
and disseminating the program. A technology of education r.equires that 
each teaching technique,} each training procedure, and each curricultim 
design procr^dure be evaluated. Identifying what works is critical in*^ 
dissemination because of the limited resources available-- school 
districts cannot afford to purchase ineffective curricula or weeks and 
weeks of consultant time. Only by identifying the techniques and 
procedures that work and discarding the others can a science of teaching 
and an affordable dissemination pikn evolve. 
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Thig chapter^ s purpose is to summarize our research concerning the 
teaching techniques, training procedures, and curriculum design 
procedures that comprise the Direct Instruction Program, The research on 
teaching techniques and training, which will be discussed first, addresses^ 
three) questions. > what extent do the teaching techniques effect Jchild 
learning? Can inexperienced teachers implement the techniques as well as 
the experienced teachersi And finally is training necessary for in- 
experienced teachers to learn the techniques? The findings concerning 
curriculum design procedures yill follow the findings on teaching^ tech- 
niques and training. 

Teaching Techniques , The teaching techniques that we have in- 

k ~ 

vestigated are pacing, corrections, reinforcement, feedback, and 
signals. Rapid pacing (see Appendix B. 1) resulted in.more correct 
responses and fewer off-task responses than slow pacing during Distar 
Reading I instruction. {When the teacher asked ajjout 12 questions per 
minute, the children answered correctly aboiit 80 pe.rcent of the time and 
were off task only ab|^t 10 percent of the t.'me. When the teacher asked 
only four questions per minute, the children answered correctly about 
30 percent of the time and were off task about 70 percent of the time.) 

Corrections following errors (see Appendix B. 2) resulted in more 
correct training and posttest responses than no corrections during 
arithmetic instruction^, (When the teacher corrected errors, the children 
answered correctly about 70 percent of the time during training and 65 
percent of the time during posttest. When the teacher did not correct 
errors, the children answered correctly about 15 percent of the time 
during training and 15 percent of the time during posttest.) In 
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■another study, (see Appendix B. 3) when children who made frequent errors 
;decoding simple words were precorrected, they read more words 'correctly. 

(When the teacher precorrected by asking the child to identify the vowel 
'sound before reading the word, the percent of words read correctly In- 

creased from 15 percent to 63. percent for one child and from 12 peyceht 

to 43 percent for the other child,) 

* . * ^ 

In the reinforcement study (see Appendix 4), social praise for 
on-task behavior increased on-task behavior, for the childVen^i (When 
the teacher praised on-task biehavior, the children were on task 
o^er 80 percent of the time. When the teacher did not praise on-task 
"behavior, the children were on task about 50 percent of the time,) In 
the feedback study (see Appendix B, 5^, children made fewer er^rors on 
their arithmetic worksheets when they were told how taany errors they 
made and graphed their performance* Since the research concerning 
reinforcement is exu^.nsive, it has received relatively little attention 
in our research program, 

A signal is a cue used 'during small group instruction to indicate 

tJjat the children are to respond in unison. In the signal *s studies . 

\ * ** . * 

(sei Appendix B. 6), teacher signals resulted in small* increases in the 
children* s attending and responding, (When the teacher used signals, the 
children attended about- 55 percent of the time and responded about 80 

[Jz&atoi the time. When the teacher did not use signals, the children 
attended about 35 percent of the time and responded about 60 percent of 
the time,) Because the effects from using signals were relatively small 
and because data relating signals to academic responding is inconclusive, 
signals as a teaching technique is being emphasized less in our training. 
Current research on acadejnic pj^rformance in a small group teaching 

2ou 
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situation l8 comparing group ''responding *and individual responding. To 

the extent that group responding enhances achievement and signals 

Increase group responding, training teachers to use signals will be a 

worthwhile training activity. ; ^ d ^ . 

Training teachers ♦ Developing an empirically based teacher training 

program requires knowledge about whicK curricula and which techniques 

enhance children's learning. The outcome data on the Distar programs 

and the teaching techniques research provide that necessary curricular 

and technique knowledge. Unfortunately the research program on teaching 

techniques has only recently demonstrated the effectiveness the 

various techniques. Since the teacher training research was dependent 

yon tlicse findings, the teacher training research is not as extensive 

.1 ' 

,as is- the research on teaching techniques. . 

; 

To ansv-r our second question — whether inexperienced teachers can 
implement the techniques as well as experienced teachers — we observed 
13 inexperienced Distar teachers twice weekly over a three-month period 
Csee Appendix B. 7). We compared the inexperienced teachers* imple- 
mentation as they received training — preservlce, inservice, and in- 
classroom — with our experiencjed teachers* implementation as measured 
during the technique evaluation research. ^ 

In the corrections study cited earlier, the teacher corrected 
approximately 90 percent of the child errors. The inexperienced 
teachers were trained to correct about 60 percent of the child errors. 
Naive teachers caa be trained to ask approximately seven questions 
per minute in contrast to the experimental teachers* rate of 12 per 
minute. Finally, the observations indicated that teachers can be 
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* 

trained to use clear signAls on approximately 75" percent of ^he. tasks.. 

V 

they present. .Although the training procedure used in this obrffepration * 

study was not evaluated^ the staff felt that the sequence for traiping tihc 

♦ 

teachers simplified the training process and was taore efficient than the 
procedures previously used in Follow Through. 

The third question was whether pi^eservice arid Ins'^rvice training 

•\ " ' , , 

were necessary for teachers to adequately pace their pi:eaentations and, . 

"' \ * ^ * 

use signals uee Appendix B. 7) . Using a multiple b^seijlned^sign 

with three teachers, appropriate signaling increased from z§ percent to 



t92 per-*.ent for one teacher, from 26 percent to 98 per^ht for anothe 
and from 29 percent to 96 percent for the third. Pa^lngS^qu^ I er 

minute) increased from 3.8 to 6.8 for one teacher, from 3.8 to 11.8^ tor 
another, and from 3.6 to 9. A for the third. 

Ahswerin>; criticisms about Distar. Some studies have been condi^ctecl 
to evaluate criticisms about the use of Distar in Follow Through. One 
criticism was that Distar wa& inappropriate fo** the higher performing 
children in Follow Through, classrooms. In one jstudy*(see\Appevidlx B. 8) . 
a classroom of high performing children who had received tvo years of 
Distar reading was evaluated in terms of achievement and attitu»^e. The ^ 
children scored almost twp years above grade level in ireadfing as measured^ 
by the Stanford Achievement Test and the* questionnaire responses in-^ 
dieted that the children enjoyed the program, thought of themselves 
as intelligent, and. had been stimulated to pursue topics introduced in 
Distar during their free time* In a Second study (see. Appendix B. 9), 
middle-ability first graders and high-ability second graders were taught 
fractions by either the Distar arithmetic procedure or by a more 
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inductive (practice only) approach. The first graders taught according 
to tne Distar protedures scored higher on two different transfer tests- 
one consisting of the same fraction skills, but with new examples, an'd 
the other consisting of untaught fraction skills* Tlie Distar second 
graders scored significantly higher on only the first transfer test. 

XKe final two studies in thi^ seqtion investigated the criticism 
that Distar involves nothing but rote learning—the children don't under- 
stand and can*t transfer vhat they learn to new problems. The first 
study focused on the extent to which Distar arithmetic teaches 
* understanding* . Learning disabled first graders were randomly assigned 
to Distar Arithmetic X instruction for one year or to a school district's 
lab ma4;h program which stressed understanding and the use of manipulatives. 
The Distar students not only, learned more arithmetic skills during the 
year but. also did as well as the lab math children in solving story 
problems that involve^d manipulatives, even though the Distar children 
never received instruction in working story problems by using manlpulatives 
The Distar children out performed the lab math children on story problems 
tha^were presented verbally. The second study (see Appendix 10) 
focused on how well children can transfer the decoding skills they learn in 
early Distar Reading .1 tb words that they have never before encountered. 
Preschoolers who uire taught by a Distar approach correctly read 92 
percent^f ^he^trar^^ preschoolers tajuight by a slight or whole- 

word approach correctly read 28 percent of the transfer words. 

Curriculum Design Procedures > Although the research in this area 
is only 'indirectly related to Follow Through *s purposes, the procedures 
under investigation were used in designing the Distar curricula and may, 
in part, explain the Engelmann-Becker Model's success. The programming 

^ tj s/ 
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pfozedures are discussed in detail b/ Becker, Engelinaim, and Thomas 
in their book on cognitive learning* The research relates to * 
five progransaing procedures;' •selecting positive and negative concept 
examples > modityrng the examples to make them easier to learn, 
sequencing the examples, providing practice, and Xeachdn^ component 
skills of complex operations before teaching the operation itself • The 
following suggestions that relate to the procedures are^ tentative and 
gqneralizations are restricted to the context of the actual studies. 

Positive and negative examples should be selected so'^iiiat students 
learn the essential* characteristics that define the concepts. Whp 
the essential characteristic assumes a range of values (e.g., the 
concept red is defined by a range of color values), .teachers should 
select positive and negative example pairs at each end of the range and 
some positive examples from within the range (see Appendix 11). 
Only, intended essential characteristics should appear in positlvte examples 

c 

Characteristics that appear in all pcsijilve examples and are absent 
from negative examples may be treated as essential by the student, 
whether a teacher intends for them to be treated that way or not (see 
Appendix B. 12). Teachers should be careful that the relevant char- 
acteristics of the negative examples are not fewer or more obvious than 
the intended essential characteristics of the positive examples; ^her- 
wise a student may learn to respond to the presence of the negative 
characteristic rather than to the presence of the positive characteristic 
(see Appendix B. 13). 

The procedures for modifying positive and negative examples are 
intended to make the essential characteristic (s) of the concept moije 
salient to the student* Students tend to leafn a concept's essential 
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a?ax:teri8tlc more* quickly il^ the characterisHc is altered 
more dissimilar from characteristics of other .examples (^e Appendi/^.B. lA).* 
Another procedure for increasing essential characteristic sali«cr, ^<Wi^g 
an emphasizer such as a coloz:, tends to quicken gaming when t^ie concept 
is defined Nby a single essential characteristic. When a concept is 

■ ^ - 

defined by several essential characteristics, emphasizerd i^ay s^low child 
acquisition (see Appendix B,, 15) • \ 

The sequencing procedures are also Intended to make essential 
characteristics more salient to the student, ror concepts defined by 
several essential characteristics, presenting positive and n*».gative 
example pairs that differ only in terms of a single essential character- 
istic results in more .rapid acquisition than presenting example pairs 
that differ in terms of several characteristics; also pair presentations 
are more ef ^ .ctive than successively presenting examples (see Appendix B. 16) 
The third sequencing procedur^ is intended to increase characteristic 
saliency by separating similar examples from each other in the order of 
introduction. The separation allows for students to discriminate 
positlye examples from negative examples that are more dissimilar* Later 
the-oore similar examples would be introduced (see Appendix B. 17). 

The practice procedure, called cumulative introduction, can lead to 
mq.re rapid acquisition of a set of concepts. In cimiulative introduction 
positive examples are introduced one at a time* and are repeatedly 
presented until the students consistently identify each, example 
introduced up to that point before new concept examples are introduced 
(see Appendix B. 17). 
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TliG final procedure is that; component skills of complex operajflons 
- 1 ' / 

are taught before the operation itself is introduced. For most students 



this procedure results in higher 



transfer scores to new prpbl^ms, | 
when it is cpmpared wifeh -just teaching the operation withmit first i 
teaching the component skills (see Appendix B. 9 and Appendix B. 10). 

Although the research suppo'rt for the selection/modification, sequencing, 
practice, and component skill teaching procedures^/i!s limited and has i 
methodological fdaws, it still/indicates that^he procedures used in j ^ - 
designing the Distar curric^a havjs some emp/irical support* Additiona.1 

studies with greater methodological rigor, 'are needed to evaluate these and 

' • / / 
Qther design procedures/exemplified, in/1)istar, A well-grounded empirical 

base for the procedilres would help account for direct instruction's ! 

effectiveness ..ad could be useful to other educators in designing 

insti^uctional programs. . j 
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